{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Interacting with Campaigns <a class=\"anchor\" id=\"top\"></a>\n",
    "\n",
    "In this notebook, you will deploy and interact with campaigns in Amazon Personalize.\n",
    "\n",
    "1. [Introduction](#intro)\n",
    "1. [Create campaigns](#create)\n",
    "1. [Interact with campaigns](#interact)\n",
    "1. [Batch recommendations](#batch)\n",
    "1. [Wrap up](#wrapup)\n",
    "\n",
    "## Introduction <a class=\"anchor\" id=\"intro\"></a>\n",
    "[Back to top](#top)\n",
    "\n",
    "At this point, you should have several solutions and at least one solution version for each. Once a solution version is created, it is possible to get recommendations from them, and to get a feel for their overall behavior.\n",
    "\n",
    "This notebook starts off by deploying each of the solution versions from the previous notebook into individual campaigns. Once they are active, there are resources for querying the recommendations, and helper functions to digest the output into something more human-readable. \n",
    "\n",
    "As you with your customer on Amazon Personalize, you can modify the helper functions to fit the structure of their data input files to keep the additional rendering working.\n",
    "\n",
    "To get started, once again, we need to import libraries, load values from previous notebooks, and load the SDK."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/05.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "from time import sleep\n",
    "import json\n",
    "from datetime import datetime\n",
    "import uuid\n",
    "import random\n",
    "\n",
    "import boto3\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%store -r"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "personalize = boto3.client('personalize')\n",
    "personalize_runtime = boto3.client('personalize-runtime')\n",
    "\n",
    "# Establish a connection to Personalize's event streaming\n",
    "personalize_events = boto3.client(service_name='personalize-events')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Interact with campaigns <a class=\"anchor\" id=\"interact\"></a>\n",
    "[Back to top](#top)\n",
    "\n",
    "Now that all campaigns are deployed and active, we can start to get recommendations via an API call. Each of the campaigns is based on a different recipe, which behave in slightly different ways because they serve different use cases. We will cover each campaign in a different order than used in previous notebooks, in order to deal with the possible complexities in ascending order (i.e. simplest first).\n",
    "\n",
    "First, let's create a supporting function to help make sense of the results returned by a Personalize campaign. Personalize returns only an `item_id`. This is great for keeping data compact, but it means you need to query a database or lookup table to get a human-readable result for the notebooks. We will create a helper function to return a human-readable result from the LastFM dataset.\n",
    "\n",
    "Start by loading in the dataset which we can use for our lookup table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>movieId</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Toy Story (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Jumanji (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Grumpier Old Men (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Waiting to Exhale (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Father of the Bride Part II (1995)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                      title\n",
       "movieId                                    \n",
       "1                          Toy Story (1995)\n",
       "2                            Jumanji (1995)\n",
       "3                   Grumpier Old Men (1995)\n",
       "4                  Waiting to Exhale (1995)\n",
       "5        Father of the Bride Part II (1995)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a dataframe for the items by reading in the correct source CSV\n",
    "items_df = pd.read_csv(dataset_dir + '/movies.csv', sep=',', usecols=[0,1], encoding='latin-1', dtype={'movieId': \"object\", 'title': \"str\"},index_col=0)\n",
    "\n",
    "# Render some sample data\n",
    "items_df.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By defining the ID column as the index column it is trivial to return an artist by just querying the ID."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Terminator 2: Judgment Day (1991)\n"
     ]
    }
   ],
   "source": [
    "movie_id_example = 589\n",
    "title = items_df.loc[movie_id_example]['title']\n",
    "print(title)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That isn't terrible, but it would get messy to repeat this everywhere in our code, so the function below will clean that up."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_movie_by_id(movie_id, movie_df=items_df):\n",
    "    \"\"\"\n",
    "    This takes in an artist_id from Personalize so it will be a string,\n",
    "    converts it to an int, and then does a lookup in a default or specified\n",
    "    dataframe.\n",
    "    \n",
    "    A really broad try/except clause was added in case anything goes wrong.\n",
    "    \n",
    "    Feel free to add more debugging or filtering here to improve results if\n",
    "    you hit an error.\n",
    "    \"\"\"\n",
    "    try:\n",
    "        return movie_df.loc[int(movie_id)]['title']\n",
    "    except:\n",
    "        return \"Error obtaining title\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's test a few simple values to check our error catching."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A known good id (The Princess Bride)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Princess Bride, The (1987)\n"
     ]
    }
   ],
   "source": [
    "print(get_movie_by_id(movie_id=\"1197\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A bad type of value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error obtaining title\n"
     ]
    }
   ],
   "source": [
    "print(get_movie_by_id(movie_id=\"987.9393939\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# _The ^^ above ^^ cell should show an error_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error obtaining title\n"
     ]
    }
   ],
   "source": [
    "# Really bad values\n",
    "print(get_movie_by_id(movie_id=\"Steve\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# _The ^^ above ^^ cell should show an error_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! Now we have a way of rendering results. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SIMS\n",
    "\n",
    "SIMS requires just an item as input, and it will return items which users interact with in similar ways to their interaction with the input item. In this particular case the item is a movie. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The cells below will handle getting recommendations from SIMS and rendering the results. Let's see what the recommendations are for the first item we looked at earlier in this notebook (Terminator 2: Judgment Day)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "get_recommendations_response = personalize_runtime.get_recommendations(\n",
    "    campaignArn = sims_campaign_arn,\n",
    "    itemId = str(589),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Jurassic Park (1993)\n",
      "Braveheart (1995)\n",
      "Terminator, The (1984)\n",
      "Fugitive, The (1993)\n",
      "Speed (1994)\n",
      "Crimson Tide (1995)\n",
      "GoldenEye (1995)\n",
      "Batman (1989)\n",
      "True Lies (1994)\n",
      "Clear and Present Danger (1994)\n",
      "Mask, The (1994)\n",
      "Die Hard: With a Vengeance (1995)\n",
      "In the Line of Fire (1993)\n",
      "Lion King, The (1994)\n",
      "Forrest Gump (1994)\n",
      "Ghost (1990)\n",
      "Apollo 13 (1995)\n",
      "Star Trek: Generations (1994)\n",
      "Cliffhanger (1993)\n",
      "Firm, The (1993)\n",
      "Die Hard (1988)\n",
      "Seven (a.k.a. Se7en) (1995)\n",
      "Indiana Jones and the Last Crusade (1989)\n",
      "Mission: Impossible (1996)\n",
      "Mrs. Doubtfire (1993)\n"
     ]
    }
   ],
   "source": [
    "item_list = get_recommendations_response['itemList']\n",
    "for item in item_list:\n",
    "    print(get_movie_by_id(movie_id=item['itemId']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Congrats, this is your first list of recommendations! This list is fine, but it would be better to see the recommendations for our sample collection of artists render in a nice dataframe. Again, let's create a helper function to achieve this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Update DF rendering\n",
    "pd.set_option('display.max_rows', 30)\n",
    "\n",
    "def get_new_recommendations_df(recommendations_df, movie_ID):\n",
    "    # Get the movie name\n",
    "    movie_name = get_movie_by_id(movie_ID)\n",
    "    # Get the recommendations\n",
    "    get_recommendations_response = personalize_runtime.get_recommendations(\n",
    "        campaignArn = sims_campaign_arn,\n",
    "        itemId = str(movie_ID),\n",
    "    )\n",
    "    # Build a new dataframe of recommendations\n",
    "    item_list = get_recommendations_response['itemList']\n",
    "    recommendation_list = []\n",
    "    for item in item_list:\n",
    "        movie = get_movie_by_id(item['itemId'])\n",
    "        recommendation_list.append(movie)\n",
    "    new_rec_DF = pd.DataFrame(recommendation_list, columns = [movie_name])\n",
    "    # Add this dataframe to the old one\n",
    "    recommendations_df = pd.concat([recommendations_df, new_rec_DF], axis=1)\n",
    "    return recommendations_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's test the helper function with several different movies. Let's sample some data from our dataset to test our SIMS campaign. Grab 5 random movies from our dataframe.\n",
    "\n",
    "Note: We are going to show similar titles, so you may want to re-run the sample until you recognize some of the movies listed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>movieId</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>59258</th>\n",
       "      <td>Baby Mama (2008)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1942</th>\n",
       "      <td>All the King's Men (1949)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1946</th>\n",
       "      <td>Marty (1955)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>84952</th>\n",
       "      <td>Confessions (Kokuhaku) (2010)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25886</th>\n",
       "      <td>Random Harvest (1942)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                 title\n",
       "movieId                               \n",
       "59258                 Baby Mama (2008)\n",
       "1942         All the King's Men (1949)\n",
       "1946                      Marty (1955)\n",
       "84952    Confessions (Kokuhaku) (2010)\n",
       "25886            Random Harvest (1942)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "samples = items_df.sample(5)\n",
    "samples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Baby Mama (2008)</th>\n",
       "      <th>All the King's Men (1949)</th>\n",
       "      <th>Marty (1955)</th>\n",
       "      <th>Confessions (Kokuhaku) (2010)</th>\n",
       "      <th>Random Harvest (1942)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Knight and Day (2010)</td>\n",
       "      <td>Attack of the Giant Leeches (1959)</td>\n",
       "      <td>Ox-Bow Incident, The (1943)</td>\n",
       "      <td>Train to Busan (2016)</td>\n",
       "      <td>High Sierra (1941)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Premonition (2007)</td>\n",
       "      <td>Asphyx, The (1973)</td>\n",
       "      <td>Dolce Vita, La (1960)</td>\n",
       "      <td>Take This Waltz (2011)</td>\n",
       "      <td>Judgment at Nuremberg (1961)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>It's Complicated (2009)</td>\n",
       "      <td>7 Faces of Dr. Lao (1964)</td>\n",
       "      <td>Long, Hot Summer, The (1958)</td>\n",
       "      <td>B/W (2015)</td>\n",
       "      <td>Wait Until Dark (1967)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Duchess, The (2008)</td>\n",
       "      <td>Atomic Cafe, The (1982)</td>\n",
       "      <td>Von Ryan's Express (1965)</td>\n",
       "      <td>War and Peace (2016)</td>\n",
       "      <td>Man Who Shot Liberty Valance, The (1962)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Revolutionary Road (2008)</td>\n",
       "      <td>5,000 Fingers of Dr. T, The (1953)</td>\n",
       "      <td>Requiem for a Heavyweight (1962)</td>\n",
       "      <td>Nymphomaniac: Volume I (2013)</td>\n",
       "      <td>Magnificent Seven, The (1960)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Casanova (2005)</td>\n",
       "      <td>Akira Kurosawa's Dreams (Dreams) (1990)</td>\n",
       "      <td>Suddenly, Last Summer (1959)</td>\n",
       "      <td>The Survivalist (2015)</td>\n",
       "      <td>Escape from Alcatraz (1979)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Nanny McPhee (2005)</td>\n",
       "      <td>Alexander Nevsky (Aleksandr Nevskiy) (1938)</td>\n",
       "      <td>Kiss Me Deadly (1955)</td>\n",
       "      <td>Scouts Guide to the Zombie Apocalypse (2015)</td>\n",
       "      <td>Shall We Dance? (Shall We Dansu?) (1996)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>August Rush (2007)</td>\n",
       "      <td>After the Thin Man (1936)</td>\n",
       "      <td>Who'll Stop the Rain (1978)</td>\n",
       "      <td>Parasyte: Part 2 (2015)</td>\n",
       "      <td>Forbidden Planet (1956)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Ju-on: The Grudge (2002)</td>\n",
       "      <td>Asphalt Jungle, The (1950)</td>\n",
       "      <td>Garden of the Finzi-Continis, The (Giardino de...</td>\n",
       "      <td>Parasyte: Part 1 (2014)</td>\n",
       "      <td>Charade (1963)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Bride Wars (2009)</td>\n",
       "      <td>20 Million Miles to Earth (1957)</td>\n",
       "      <td>Happy Accidents (2000)</td>\n",
       "      <td>Knockin' on Heaven's Door (1997)</td>\n",
       "      <td>Mississippi Burning (1988)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Enough (2002)</td>\n",
       "      <td>Another 48 Hrs. (1990)</td>\n",
       "      <td>Our Lady of the Assassins (Virgen de los sicar...</td>\n",
       "      <td>Demolition (2016)</td>\n",
       "      <td>High Noon (1952)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Sunshine Cleaning (2008)</td>\n",
       "      <td>Animal Crackers (1930)</td>\n",
       "      <td>Rape Me (Baise-moi) (2000)</td>\n",
       "      <td>The Night Before (2015)</td>\n",
       "      <td>To Catch a Thief (1955)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Producers, The (2005)</td>\n",
       "      <td>Attack of the Puppet People (1958)</td>\n",
       "      <td>Anniversary Party, The (2001)</td>\n",
       "      <td>The Secret Life of Pets (2016)</td>\n",
       "      <td>Master and Commander: The Far Side of the Worl...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>She's the Man (2006)</td>\n",
       "      <td>Atomic Brain, The (1963)</td>\n",
       "      <td>Written on the Wind (1956)</td>\n",
       "      <td>Me and Earl and the Dying Girl (2015)</td>\n",
       "      <td>Arsenic and Old Lace (1944)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>The Amazing Spider-Man 2 (2014)</td>\n",
       "      <td>Amazing Transparent Man, The (1960)</td>\n",
       "      <td>Misfits, The (1961)</td>\n",
       "      <td>Dope (2015)</td>\n",
       "      <td>Dirty Dozen, The (1967)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Penelope (2006)</td>\n",
       "      <td>Atragon (Kaitei Gunkan) (1963)</td>\n",
       "      <td>My Man Godfrey (1936)</td>\n",
       "      <td>Hot Tub Time Machine (2010)</td>\n",
       "      <td>Philadelphia Story, The (1940)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Princess Diaries 2: Royal Engagement, The (2004)</td>\n",
       "      <td>Alien from L.A. (1988)</td>\n",
       "      <td>Meet John Doe (1941)</td>\n",
       "      <td>Ghostbusters (2016)</td>\n",
       "      <td>Patton (1970)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Marie Antoinette (2006)</td>\n",
       "      <td>Alice in Wonderland (1933)</td>\n",
       "      <td>Phantom of the Paradise (1974)</td>\n",
       "      <td>Postman, The (1997)</td>\n",
       "      <td>Great Escape, The (1963)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Valentine's Day (2010)</td>\n",
       "      <td>7th Voyage of Sinbad, The (1958)</td>\n",
       "      <td>All That Heaven Allows (1955)</td>\n",
       "      <td>SLC Punk! (1998)</td>\n",
       "      <td>Seven Samurai (Shichinin no samurai) (1954)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Ugly Truth, The (2009)</td>\n",
       "      <td>Angry Red Planet, The (1959)</td>\n",
       "      <td>Brother (2000)</td>\n",
       "      <td>Vicky Cristina Barcelona (2008)</td>\n",
       "      <td>Sting, The (1973)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Radio (2003)</td>\n",
       "      <td>American Friend, The (Amerikanische Freund, De...</td>\n",
       "      <td>Sweet Smell of Success (1957)</td>\n",
       "      <td>21 Jump Street (2012)</td>\n",
       "      <td>North by Northwest (1959)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Crazy Heart (2009)</td>\n",
       "      <td>American Grindhouse (2010)</td>\n",
       "      <td>L.I.E. (2001)</td>\n",
       "      <td>Hard Candy (2005)</td>\n",
       "      <td>Wizard of Oz, The (1939)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Lincoln (2012)</td>\n",
       "      <td>Agony and the Ecstasy, The (1965)</td>\n",
       "      <td>Pledge, The (2001)</td>\n",
       "      <td>Dawn of the Planet of the Apes (2014)</td>\n",
       "      <td>Jaws (1975)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Time Traveler's Wife, The (2009)</td>\n",
       "      <td>And Starring Pancho Villa as Himself (2003)</td>\n",
       "      <td>Theremin: An Electronic Odyssey (1993)</td>\n",
       "      <td>127 Hours (2010)</td>\n",
       "      <td>Rear Window (1954)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Jack Reacher (2012)</td>\n",
       "      <td>Attack of the 50 Foot Woman (1958)</td>\n",
       "      <td>Cries and Whispers (Viskningar och rop) (1972)</td>\n",
       "      <td>Happiness (1998)</td>\n",
       "      <td>Life Is Beautiful (La Vita Ã¨ bella) (1997)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                    Baby Mama (2008)  \\\n",
       "0                              Knight and Day (2010)   \n",
       "1                                 Premonition (2007)   \n",
       "2                            It's Complicated (2009)   \n",
       "3                                Duchess, The (2008)   \n",
       "4                          Revolutionary Road (2008)   \n",
       "5                                    Casanova (2005)   \n",
       "6                                Nanny McPhee (2005)   \n",
       "7                                 August Rush (2007)   \n",
       "8                           Ju-on: The Grudge (2002)   \n",
       "9                                  Bride Wars (2009)   \n",
       "10                                     Enough (2002)   \n",
       "11                          Sunshine Cleaning (2008)   \n",
       "12                             Producers, The (2005)   \n",
       "13                              She's the Man (2006)   \n",
       "14                   The Amazing Spider-Man 2 (2014)   \n",
       "15                                   Penelope (2006)   \n",
       "16  Princess Diaries 2: Royal Engagement, The (2004)   \n",
       "17                           Marie Antoinette (2006)   \n",
       "18                            Valentine's Day (2010)   \n",
       "19                            Ugly Truth, The (2009)   \n",
       "20                                      Radio (2003)   \n",
       "21                                Crazy Heart (2009)   \n",
       "22                                    Lincoln (2012)   \n",
       "23                  Time Traveler's Wife, The (2009)   \n",
       "24                               Jack Reacher (2012)   \n",
       "\n",
       "                            All the King's Men (1949)  \\\n",
       "0                  Attack of the Giant Leeches (1959)   \n",
       "1                                  Asphyx, The (1973)   \n",
       "2                           7 Faces of Dr. Lao (1964)   \n",
       "3                             Atomic Cafe, The (1982)   \n",
       "4                  5,000 Fingers of Dr. T, The (1953)   \n",
       "5             Akira Kurosawa's Dreams (Dreams) (1990)   \n",
       "6         Alexander Nevsky (Aleksandr Nevskiy) (1938)   \n",
       "7                           After the Thin Man (1936)   \n",
       "8                          Asphalt Jungle, The (1950)   \n",
       "9                    20 Million Miles to Earth (1957)   \n",
       "10                             Another 48 Hrs. (1990)   \n",
       "11                             Animal Crackers (1930)   \n",
       "12                 Attack of the Puppet People (1958)   \n",
       "13                           Atomic Brain, The (1963)   \n",
       "14                Amazing Transparent Man, The (1960)   \n",
       "15                     Atragon (Kaitei Gunkan) (1963)   \n",
       "16                             Alien from L.A. (1988)   \n",
       "17                         Alice in Wonderland (1933)   \n",
       "18                   7th Voyage of Sinbad, The (1958)   \n",
       "19                       Angry Red Planet, The (1959)   \n",
       "20  American Friend, The (Amerikanische Freund, De...   \n",
       "21                         American Grindhouse (2010)   \n",
       "22                  Agony and the Ecstasy, The (1965)   \n",
       "23        And Starring Pancho Villa as Himself (2003)   \n",
       "24                 Attack of the 50 Foot Woman (1958)   \n",
       "\n",
       "                                         Marty (1955)  \\\n",
       "0                         Ox-Bow Incident, The (1943)   \n",
       "1                               Dolce Vita, La (1960)   \n",
       "2                        Long, Hot Summer, The (1958)   \n",
       "3                           Von Ryan's Express (1965)   \n",
       "4                    Requiem for a Heavyweight (1962)   \n",
       "5                        Suddenly, Last Summer (1959)   \n",
       "6                               Kiss Me Deadly (1955)   \n",
       "7                         Who'll Stop the Rain (1978)   \n",
       "8   Garden of the Finzi-Continis, The (Giardino de...   \n",
       "9                              Happy Accidents (2000)   \n",
       "10  Our Lady of the Assassins (Virgen de los sicar...   \n",
       "11                         Rape Me (Baise-moi) (2000)   \n",
       "12                      Anniversary Party, The (2001)   \n",
       "13                         Written on the Wind (1956)   \n",
       "14                                Misfits, The (1961)   \n",
       "15                              My Man Godfrey (1936)   \n",
       "16                               Meet John Doe (1941)   \n",
       "17                     Phantom of the Paradise (1974)   \n",
       "18                      All That Heaven Allows (1955)   \n",
       "19                                     Brother (2000)   \n",
       "20                      Sweet Smell of Success (1957)   \n",
       "21                                      L.I.E. (2001)   \n",
       "22                                 Pledge, The (2001)   \n",
       "23             Theremin: An Electronic Odyssey (1993)   \n",
       "24     Cries and Whispers (Viskningar och rop) (1972)   \n",
       "\n",
       "                   Confessions (Kokuhaku) (2010)  \\\n",
       "0                          Train to Busan (2016)   \n",
       "1                         Take This Waltz (2011)   \n",
       "2                                     B/W (2015)   \n",
       "3                           War and Peace (2016)   \n",
       "4                  Nymphomaniac: Volume I (2013)   \n",
       "5                         The Survivalist (2015)   \n",
       "6   Scouts Guide to the Zombie Apocalypse (2015)   \n",
       "7                        Parasyte: Part 2 (2015)   \n",
       "8                        Parasyte: Part 1 (2014)   \n",
       "9               Knockin' on Heaven's Door (1997)   \n",
       "10                             Demolition (2016)   \n",
       "11                       The Night Before (2015)   \n",
       "12                The Secret Life of Pets (2016)   \n",
       "13         Me and Earl and the Dying Girl (2015)   \n",
       "14                                   Dope (2015)   \n",
       "15                   Hot Tub Time Machine (2010)   \n",
       "16                           Ghostbusters (2016)   \n",
       "17                           Postman, The (1997)   \n",
       "18                              SLC Punk! (1998)   \n",
       "19               Vicky Cristina Barcelona (2008)   \n",
       "20                         21 Jump Street (2012)   \n",
       "21                             Hard Candy (2005)   \n",
       "22         Dawn of the Planet of the Apes (2014)   \n",
       "23                              127 Hours (2010)   \n",
       "24                              Happiness (1998)   \n",
       "\n",
       "                                Random Harvest (1942)  \n",
       "0                                  High Sierra (1941)  \n",
       "1                        Judgment at Nuremberg (1961)  \n",
       "2                              Wait Until Dark (1967)  \n",
       "3            Man Who Shot Liberty Valance, The (1962)  \n",
       "4                       Magnificent Seven, The (1960)  \n",
       "5                         Escape from Alcatraz (1979)  \n",
       "6            Shall We Dance? (Shall We Dansu?) (1996)  \n",
       "7                             Forbidden Planet (1956)  \n",
       "8                                      Charade (1963)  \n",
       "9                          Mississippi Burning (1988)  \n",
       "10                                   High Noon (1952)  \n",
       "11                            To Catch a Thief (1955)  \n",
       "12  Master and Commander: The Far Side of the Worl...  \n",
       "13                        Arsenic and Old Lace (1944)  \n",
       "14                            Dirty Dozen, The (1967)  \n",
       "15                     Philadelphia Story, The (1940)  \n",
       "16                                      Patton (1970)  \n",
       "17                           Great Escape, The (1963)  \n",
       "18        Seven Samurai (Shichinin no samurai) (1954)  \n",
       "19                                  Sting, The (1973)  \n",
       "20                          North by Northwest (1959)  \n",
       "21                           Wizard of Oz, The (1939)  \n",
       "22                                        Jaws (1975)  \n",
       "23                                 Rear Window (1954)  \n",
       "24        Life Is Beautiful (La Vita Ã¨ bella) (1997)  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sims_recommendations_df = pd.DataFrame()\n",
    "movies = samples.index.tolist()\n",
    "\n",
    "for movie in movies:\n",
    "    sims_recommendations_df = get_new_recommendations_df(sims_recommendations_df, movie)\n",
    "\n",
    "sims_recommendations_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may notice that a lot of the items look the same, hopefully not all of them do (this is more likely with a smaller # of interactions). This shows that the evaluation metrics should not be the only thing you rely on when evaluating your solution version. So when this happens, what can you do to improve the results?\n",
    "\n",
    "This is a good time to think about the hyperparameters of the Personalize recipes. The SIMS recipe has a `popularity_discount_factor` hyperparameter (see [documentation](https://docs.aws.amazon.com/personalize/latest/dg/native-recipe-sims.html)). Leveraging this hyperparameter allows you to control the nuance you see in the results. This parameter and its behavior will be unique to every dataset you encounter, and depends on the goals of the business. You can iterate on the value of this hyperparameter until you are satisfied with the results, or you can start by leveraging Personalize's hyperparameter optimization (HPO) feature. For more information on hyperparameters and HPO tuning, see the [documentation](https://docs.aws.amazon.com/personalize/latest/dg/customizing-solution-config-hpo.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### User Personalization\n",
    "\n",
    "HRNN is one of the more advanced algorithms provided by Amazon Personalize. It supports personalization of the items for a specific user based on their past behavior and can intake real time events in order to alter recommendations for a user without retraining. \n",
    "\n",
    "Since HRNN relies on having a sampling of users, let's load the data we need for that and select 3 random users. Since Movielens does not include user data, we will select 3 random numbers from the range of user id's in the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[164, 563, 396]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "if not USE_FULL_MOVIELENS:\n",
    "    users = random.sample(range(1, 600), 3)\n",
    "else:\n",
    "    users = random.sample(range(1, 162000), 3)\n",
    "users"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we render the recommendations for our 3 random users from above. After that, we will explore real-time interactions before moving on to Personalized Ranking.\n",
    "\n",
    "Again, we create a helper function to render the results in a nice dataframe.\n",
    "\n",
    "#### API call results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Update DF rendering\n",
    "pd.set_option('display.max_rows', 30)\n",
    "\n",
    "def get_new_recommendations_df_users(recommendations_df, user_id):\n",
    "    # Get the movie name\n",
    "    #movie_name = get_movie_by_id(artist_ID)\n",
    "    # Get the recommendations\n",
    "    get_recommendations_response = personalize_runtime.get_recommendations(\n",
    "        campaignArn = userpersonalization_campaign_arn,\n",
    "        userId = str(user_id),\n",
    "    )\n",
    "    # Build a new dataframe of recommendations\n",
    "    item_list = get_recommendations_response['itemList']\n",
    "    recommendation_list = []\n",
    "    for item in item_list:\n",
    "        movie = get_movie_by_id(item['itemId'])\n",
    "        recommendation_list.append(movie)\n",
    "    #print(recommendation_list)\n",
    "    new_rec_DF = pd.DataFrame(recommendation_list, columns = [user_id])\n",
    "    # Add this dataframe to the old one\n",
    "    recommendations_df = pd.concat([recommendations_df, new_rec_DF], axis=1)\n",
    "    return recommendations_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>164</th>\n",
       "      <th>563</th>\n",
       "      <th>396</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Superman II (1980)</td>\n",
       "      <td>Fargo (1996)</td>\n",
       "      <td>Shrek (2001)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Star Trek II: The Wrath of Khan (1982)</td>\n",
       "      <td>Yes Man (2008)</td>\n",
       "      <td>Amelie (Fabuleux destin d'AmÃ©lie Poulain, Le)...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Running Man, The (1987)</td>\n",
       "      <td>Holiday, The (2006)</td>\n",
       "      <td>Lord of the Rings: The Two Towers, The (2002)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Star Trek V: The Final Frontier (1989)</td>\n",
       "      <td>Borat: Cultural Learnings of America for Make ...</td>\n",
       "      <td>Toy Story 2 (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Planet of the Apes (1968)</td>\n",
       "      <td>Devil Wears Prada, The (2006)</td>\n",
       "      <td>Good Will Hunting (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Star Trek VI: The Undiscovered Country (1991)</td>\n",
       "      <td>Notting Hill (1999)</td>\n",
       "      <td>Eternal Sunshine of the Spotless Mind (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Terminator, The (1984)</td>\n",
       "      <td>The Imitation Game (2014)</td>\n",
       "      <td>Spirited Away (Sen to Chihiro no kamikakushi) ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Tron (1982)</td>\n",
       "      <td>Hurt Locker, The (2008)</td>\n",
       "      <td>Lord of the Rings: The Return of the King, The...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Star Trek III: The Search for Spock (1984)</td>\n",
       "      <td>Hitch (2005)</td>\n",
       "      <td>Schindler's List (1993)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>RoboCop (1987)</td>\n",
       "      <td>Mean Girls (2004)</td>\n",
       "      <td>LÃ©on: The Professional (a.k.a. The Profession...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Abyss, The (1989)</td>\n",
       "      <td>Forgetting Sarah Marshall (2008)</td>\n",
       "      <td>Beautiful Mind, A (2001)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Superman (1978)</td>\n",
       "      <td>In Bruges (2008)</td>\n",
       "      <td>Monsters, Inc. (2001)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Rocketeer, The (1991)</td>\n",
       "      <td>Hangover, The (2009)</td>\n",
       "      <td>Life Is Beautiful (La Vita Ã¨ bella) (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Total Recall (1990)</td>\n",
       "      <td>Wedding Crashers (2005)</td>\n",
       "      <td>Forrest Gump (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Star Trek IV: The Voyage Home (1986)</td>\n",
       "      <td>(500) Days of Summer (2009)</td>\n",
       "      <td>Godfather, The (1972)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Star Trek: The Motion Picture (1979)</td>\n",
       "      <td>Zombieland (2009)</td>\n",
       "      <td>Finding Nemo (2003)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Logan's Run (1976)</td>\n",
       "      <td>Up in the Air (2009)</td>\n",
       "      <td>Lion King, The (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Top Gun (1986)</td>\n",
       "      <td>Girl with the Dragon Tattoo, The (MÃ¤n som hat...</td>\n",
       "      <td>American Beauty (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Romancing the Stone (1984)</td>\n",
       "      <td>Proposal, The (2009)</td>\n",
       "      <td>Lord of the Rings: The Fellowship of the Ring,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Back to the Future Part III (1990)</td>\n",
       "      <td>Life Is Beautiful (La Vita Ã¨ bella) (1997)</td>\n",
       "      <td>2001: A Space Odyssey (1968)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Escape from New York (1981)</td>\n",
       "      <td>Knocked Up (2007)</td>\n",
       "      <td>Pirates of the Caribbean: The Curse of the Bla...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Road Warrior, The (Mad Max 2) (1981)</td>\n",
       "      <td>Deadpool (2016)</td>\n",
       "      <td>Apollo 13 (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Thing, The (1982)</td>\n",
       "      <td>Prestige, The (2006)</td>\n",
       "      <td>Pulp Fiction (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Willow (1988)</td>\n",
       "      <td>Idiocracy (2006)</td>\n",
       "      <td>Harry Potter and the Prisoner of Azkaban (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Ghostbusters (a.k.a. Ghost Busters) (1984)</td>\n",
       "      <td>Anchorman: The Legend of Ron Burgundy (2004)</td>\n",
       "      <td>City of God (Cidade de Deus) (2002)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                              164  \\\n",
       "0                              Superman II (1980)   \n",
       "1          Star Trek II: The Wrath of Khan (1982)   \n",
       "2                         Running Man, The (1987)   \n",
       "3          Star Trek V: The Final Frontier (1989)   \n",
       "4                       Planet of the Apes (1968)   \n",
       "5   Star Trek VI: The Undiscovered Country (1991)   \n",
       "6                          Terminator, The (1984)   \n",
       "7                                     Tron (1982)   \n",
       "8      Star Trek III: The Search for Spock (1984)   \n",
       "9                                  RoboCop (1987)   \n",
       "10                              Abyss, The (1989)   \n",
       "11                                Superman (1978)   \n",
       "12                          Rocketeer, The (1991)   \n",
       "13                            Total Recall (1990)   \n",
       "14           Star Trek IV: The Voyage Home (1986)   \n",
       "15           Star Trek: The Motion Picture (1979)   \n",
       "16                             Logan's Run (1976)   \n",
       "17                                 Top Gun (1986)   \n",
       "18                     Romancing the Stone (1984)   \n",
       "19             Back to the Future Part III (1990)   \n",
       "20                    Escape from New York (1981)   \n",
       "21           Road Warrior, The (Mad Max 2) (1981)   \n",
       "22                              Thing, The (1982)   \n",
       "23                                  Willow (1988)   \n",
       "24     Ghostbusters (a.k.a. Ghost Busters) (1984)   \n",
       "\n",
       "                                                  563  \\\n",
       "0                                        Fargo (1996)   \n",
       "1                                      Yes Man (2008)   \n",
       "2                                 Holiday, The (2006)   \n",
       "3   Borat: Cultural Learnings of America for Make ...   \n",
       "4                       Devil Wears Prada, The (2006)   \n",
       "5                                 Notting Hill (1999)   \n",
       "6                           The Imitation Game (2014)   \n",
       "7                             Hurt Locker, The (2008)   \n",
       "8                                        Hitch (2005)   \n",
       "9                                   Mean Girls (2004)   \n",
       "10                   Forgetting Sarah Marshall (2008)   \n",
       "11                                   In Bruges (2008)   \n",
       "12                               Hangover, The (2009)   \n",
       "13                            Wedding Crashers (2005)   \n",
       "14                        (500) Days of Summer (2009)   \n",
       "15                                  Zombieland (2009)   \n",
       "16                               Up in the Air (2009)   \n",
       "17  Girl with the Dragon Tattoo, The (MÃ¤n som hat...   \n",
       "18                               Proposal, The (2009)   \n",
       "19        Life Is Beautiful (La Vita Ã¨ bella) (1997)   \n",
       "20                                  Knocked Up (2007)   \n",
       "21                                    Deadpool (2016)   \n",
       "22                               Prestige, The (2006)   \n",
       "23                                   Idiocracy (2006)   \n",
       "24       Anchorman: The Legend of Ron Burgundy (2004)   \n",
       "\n",
       "                                                  396  \n",
       "0                                        Shrek (2001)  \n",
       "1   Amelie (Fabuleux destin d'AmÃ©lie Poulain, Le)...  \n",
       "2       Lord of the Rings: The Two Towers, The (2002)  \n",
       "3                                  Toy Story 2 (1999)  \n",
       "4                            Good Will Hunting (1997)  \n",
       "5        Eternal Sunshine of the Spotless Mind (2004)  \n",
       "6   Spirited Away (Sen to Chihiro no kamikakushi) ...  \n",
       "7   Lord of the Rings: The Return of the King, The...  \n",
       "8                             Schindler's List (1993)  \n",
       "9   LÃ©on: The Professional (a.k.a. The Profession...  \n",
       "10                           Beautiful Mind, A (2001)  \n",
       "11                              Monsters, Inc. (2001)  \n",
       "12        Life Is Beautiful (La Vita Ã¨ bella) (1997)  \n",
       "13                                Forrest Gump (1994)  \n",
       "14                              Godfather, The (1972)  \n",
       "15                                Finding Nemo (2003)  \n",
       "16                              Lion King, The (1994)  \n",
       "17                             American Beauty (1999)  \n",
       "18  Lord of the Rings: The Fellowship of the Ring,...  \n",
       "19                       2001: A Space Odyssey (1968)  \n",
       "20  Pirates of the Caribbean: The Curse of the Bla...  \n",
       "21                                   Apollo 13 (1995)  \n",
       "22                                Pulp Fiction (1994)  \n",
       "23    Harry Potter and the Prisoner of Azkaban (2004)  \n",
       "24                City of God (Cidade de Deus) (2002)  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recommendations_df_users = pd.DataFrame()\n",
    "#users = users_df.sample(3).index.tolist()\n",
    "\n",
    "for user in users:\n",
    "    recommendations_df_users = get_new_recommendations_df_users(recommendations_df_users, user)\n",
    "\n",
    "recommendations_df_users"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we clearly see that the recommendations for each user are different. If you were to need a cache for these results, you could start by running the API calls through all your users and store the results, or you could use a batch export, which will be covered later in this notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now lets apply item filters to see recommendations for one of these users within a genre\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_new_recommendations_df_by_filter(recommendations_df, user_id, filter_arn):\n",
    "    # Get the movie name\n",
    "    #movie_name = get_movie_by_id(artist_ID)\n",
    "    # Get the recommendations\n",
    "    get_recommendations_response = personalize_runtime.get_recommendations(\n",
    "        campaignArn = userpersonalization_campaign_arn,\n",
    "        userId = str(user_id),\n",
    "        filterArn = filter_arn\n",
    "    )\n",
    "    # Build a new dataframe of recommendations\n",
    "    item_list = get_recommendations_response['itemList']\n",
    "    recommendation_list = []\n",
    "    for item in item_list:\n",
    "        movie = get_movie_by_id(item['itemId'])\n",
    "        recommendation_list.append(movie)\n",
    "    #print(recommendation_list)\n",
    "    filter_name = filter_arn.split('/')[1]\n",
    "    new_rec_DF = pd.DataFrame(recommendation_list, columns = [filter_name])\n",
    "    # Add this dataframe to the old one\n",
    "    recommendations_df = pd.concat([recommendations_df, new_rec_DF], axis=1)\n",
    "    return recommendations_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "396"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see the recommendations for movies within a given genre. Within a VOD application you could create Shelves (also known as rails or carosels) easily by using these filters. Depending on the information you have about your items, You could also filter on additional information such as keyword, year/decade etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Animation</th>\n",
       "      <th>Drama</th>\n",
       "      <th>Children</th>\n",
       "      <th>Crime</th>\n",
       "      <th>Film-Noir</th>\n",
       "      <th>Mystery</th>\n",
       "      <th>Sci-Fi</th>\n",
       "      <th>1970s</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Shrek (2001)</td>\n",
       "      <td>Good Will Hunting (1997)</td>\n",
       "      <td>Shrek (2001)</td>\n",
       "      <td>LÃ©on: The Professional (a.k.a. The Profession...</td>\n",
       "      <td>Mulholland Drive (2001)</td>\n",
       "      <td>Donnie Darko (2001)</td>\n",
       "      <td>Eternal Sunshine of the Spotless Mind (2004)</td>\n",
       "      <td>Godfather, The (1972)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Toy Story 2 (1999)</td>\n",
       "      <td>Eternal Sunshine of the Spotless Mind (2004)</td>\n",
       "      <td>Toy Story 2 (1999)</td>\n",
       "      <td>Godfather, The (1972)</td>\n",
       "      <td>Chinatown (1974)</td>\n",
       "      <td>Sixth Sense, The (1999)</td>\n",
       "      <td>2001: A Space Odyssey (1968)</td>\n",
       "      <td>One Flew Over the Cuckoo's Nest (1975)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Spirited Away (Sen to Chihiro no kamikakushi) ...</td>\n",
       "      <td>Lord of the Rings: The Return of the King, The...</td>\n",
       "      <td>Monsters, Inc. (2001)</td>\n",
       "      <td>Pulp Fiction (1994)</td>\n",
       "      <td>L.A. Confidential (1997)</td>\n",
       "      <td>Game, The (1997)</td>\n",
       "      <td>Donnie Darko (2001)</td>\n",
       "      <td>Taxi Driver (1976)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Monsters, Inc. (2001)</td>\n",
       "      <td>Schindler's List (1993)</td>\n",
       "      <td>Finding Nemo (2003)</td>\n",
       "      <td>City of God (Cidade de Deus) (2002)</td>\n",
       "      <td>Notorious (1946)</td>\n",
       "      <td>Twelve Monkeys (a.k.a. 12 Monkeys) (1995)</td>\n",
       "      <td>Matrix Reloaded, The (2003)</td>\n",
       "      <td>Clockwork Orange, A (1971)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Finding Nemo (2003)</td>\n",
       "      <td>LÃ©on: The Professional (a.k.a. The Profession...</td>\n",
       "      <td>Lion King, The (1994)</td>\n",
       "      <td>Fargo (1996)</td>\n",
       "      <td>Sin City (2005)</td>\n",
       "      <td>City of Lost Children, The (CitÃ© des enfants ...</td>\n",
       "      <td>X-Men (2000)</td>\n",
       "      <td>Monty Python and the Holy Grail (1975)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Lion King, The (1994)</td>\n",
       "      <td>Beautiful Mind, A (2001)</td>\n",
       "      <td>Incredibles, The (2004)</td>\n",
       "      <td>O Brother, Where Art Thou? (2000)</td>\n",
       "      <td>Strangers on a Train (1951)</td>\n",
       "      <td>Vertigo (1958)</td>\n",
       "      <td>Star Wars: Episode VI - Return of the Jedi (1983)</td>\n",
       "      <td>Star Wars: Episode IV - A New Hope (1977)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Incredibles, The (2004)</td>\n",
       "      <td>Life Is Beautiful (La Vita Ã¨ bella) (1997)</td>\n",
       "      <td>Shrek 2 (2004)</td>\n",
       "      <td>Green Mile, The (1999)</td>\n",
       "      <td>Dark City (1998)</td>\n",
       "      <td>Who Framed Roger Rabbit? (1988)</td>\n",
       "      <td>Matrix Revolutions, The (2003)</td>\n",
       "      <td>Godfather: Part II, The (1974)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Wallace &amp; Gromit: The Best of Aardman Animatio...</td>\n",
       "      <td>Forrest Gump (1994)</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Taxi Driver (1976)</td>\n",
       "      <td>Sunset Blvd. (a.k.a. Sunset Boulevard) (1950)</td>\n",
       "      <td>Memento (2000)</td>\n",
       "      <td>Clockwork Orange, A (1971)</td>\n",
       "      <td>Apocalypse Now (1979)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Shrek 2 (2004)</td>\n",
       "      <td>Godfather, The (1972)</td>\n",
       "      <td>Beauty and the Beast (1991)</td>\n",
       "      <td>Catch Me If You Can (2002)</td>\n",
       "      <td>Miller's Crossing (1990)</td>\n",
       "      <td>Reservoir Dogs (1992)</td>\n",
       "      <td>Independence Day (a.k.a. ID4) (1996)</td>\n",
       "      <td>Robin Hood (1973)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Lion King, The (1994)</td>\n",
       "      <td>Up (2009)</td>\n",
       "      <td>Trainspotting (1996)</td>\n",
       "      <td>Key Largo (1948)</td>\n",
       "      <td>Minority Report (2002)</td>\n",
       "      <td>Back to the Future (1985)</td>\n",
       "      <td>Willy Wonka &amp; the Chocolate Factory (1971)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Howl's Moving Castle (Hauru no ugoku shiro) (2...</td>\n",
       "      <td>American Beauty (1999)</td>\n",
       "      <td>Harry Potter and the Sorcerer's Stone (a.k.a. ...</td>\n",
       "      <td>Shawshank Redemption, The (1994)</td>\n",
       "      <td>Double Indemnity (1944)</td>\n",
       "      <td>Seven (a.k.a. Se7en) (1995)</td>\n",
       "      <td>V for Vendetta (2006)</td>\n",
       "      <td>American Graffiti (1973)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Beauty and the Beast (1991)</td>\n",
       "      <td>2001: A Space Odyssey (1968)</td>\n",
       "      <td>Wallace &amp; Gromit: A Close Shave (1995)</td>\n",
       "      <td>Departed, The (2006)</td>\n",
       "      <td>While the City Sleeps (1956)</td>\n",
       "      <td>The Machinist (2004)</td>\n",
       "      <td>Twelve Monkeys (a.k.a. 12 Monkeys) (1995)</td>\n",
       "      <td>Sting, The (1973)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Princess Mononoke (Mononoke-hime) (1997)</td>\n",
       "      <td>Apollo 13 (1995)</td>\n",
       "      <td>Chronicles of Narnia: The Lion, the Witch and ...</td>\n",
       "      <td>Crash (2004)</td>\n",
       "      <td>Night of the Hunter, The (1955)</td>\n",
       "      <td>Usual Suspects, The (1995)</td>\n",
       "      <td>Star Wars: Episode IV - A New Hope (1977)</td>\n",
       "      <td>Alien (1979)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Up (2009)</td>\n",
       "      <td>Pulp Fiction (1994)</td>\n",
       "      <td>Grand Day Out with Wallace and Gromit, A (1989)</td>\n",
       "      <td>Clockwork Orange, A (1971)</td>\n",
       "      <td>Lost Highway (1997)</td>\n",
       "      <td>North by Northwest (1959)</td>\n",
       "      <td>City of Lost Children, The (CitÃ© des enfants ...</td>\n",
       "      <td>Monty Python's Life of Brian (1979)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Wallace &amp; Gromit: A Close Shave (1995)</td>\n",
       "      <td>City of God (Cidade de Deus) (2002)</td>\n",
       "      <td>Iron Giant, The (1999)</td>\n",
       "      <td>Silence of the Lambs, The (1991)</td>\n",
       "      <td>Touch of Evil (1958)</td>\n",
       "      <td>Prestige, The (2006)</td>\n",
       "      <td>Grand Day Out with Wallace and Gromit, A (1989)</td>\n",
       "      <td>Deliverance (1972)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Grand Day Out with Wallace and Gromit, A (1989)</td>\n",
       "      <td>Donnie Darko (2001)</td>\n",
       "      <td>Babe (1995)</td>\n",
       "      <td>Dark Knight, The (2008)</td>\n",
       "      <td>Stray Dog (Nora inu) (1949)</td>\n",
       "      <td>Inception (2010)</td>\n",
       "      <td>Iron Giant, The (1999)</td>\n",
       "      <td>Young Frankenstein (1974)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Iron Giant, The (1999)</td>\n",
       "      <td>Crouching Tiger, Hidden Dragon (Wo hu cang lon...</td>\n",
       "      <td>Who Framed Roger Rabbit? (1988)</td>\n",
       "      <td>Fight Club (1999)</td>\n",
       "      <td>M (1931)</td>\n",
       "      <td>Citizen Kane (1941)</td>\n",
       "      <td>WALLÂ·E (2008)</td>\n",
       "      <td>Aguirre: The Wrath of God (Aguirre, der Zorn G...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Who Framed Roger Rabbit? (1988)</td>\n",
       "      <td>Fargo (1996)</td>\n",
       "      <td>WALLÂ·E (2008)</td>\n",
       "      <td>American History X (1998)</td>\n",
       "      <td>Dark Passage (1947)</td>\n",
       "      <td>Rear Window (1954)</td>\n",
       "      <td>Spider-Man 2 (2004)</td>\n",
       "      <td>Chinatown (1974)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>WALLÂ·E (2008)</td>\n",
       "      <td>Erin Brockovich (2000)</td>\n",
       "      <td>Charlie and the Chocolate Factory (2005)</td>\n",
       "      <td>Godfather: Part II, The (1974)</td>\n",
       "      <td>Call Northside 777 (1948)</td>\n",
       "      <td>Eyes Wide Shut (1999)</td>\n",
       "      <td>NausicaÃ¤ of the Valley of the Wind (Kaze no t...</td>\n",
       "      <td>M*A*S*H (a.k.a. MASH) (1970)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>NausicaÃ¤ of the Valley of the Wind (Kaze no t...</td>\n",
       "      <td>Cast Away (2000)</td>\n",
       "      <td>Bug's Life, A (1998)</td>\n",
       "      <td>Who Framed Roger Rabbit? (1988)</td>\n",
       "      <td>High Sierra (1941)</td>\n",
       "      <td>Gosford Park (2001)</td>\n",
       "      <td>Contact (1997)</td>\n",
       "      <td>Close Encounters of the Third Kind (1977)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Bug's Life, A (1998)</td>\n",
       "      <td>To Kill a Mockingbird (1962)</td>\n",
       "      <td>Aladdin (1992)</td>\n",
       "      <td>Goodfellas (1990)</td>\n",
       "      <td>Gilda (1946)</td>\n",
       "      <td>Mystic River (2003)</td>\n",
       "      <td>Truman Show, The (1998)</td>\n",
       "      <td>Deer Hunter, The (1978)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Aladdin (1992)</td>\n",
       "      <td>Saving Private Ryan (1998)</td>\n",
       "      <td>Wallace &amp; Gromit: The Wrong Trousers (1993)</td>\n",
       "      <td>Wallace &amp; Gromit: The Wrong Trousers (1993)</td>\n",
       "      <td>Johnny Eager (1942)</td>\n",
       "      <td>Mulholland Drive (2001)</td>\n",
       "      <td>X2: X-Men United (2003)</td>\n",
       "      <td>Badlands (1973)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Wallace &amp; Gromit: The Wrong Trousers (1993)</td>\n",
       "      <td>Big Fish (2003)</td>\n",
       "      <td>My Neighbor Totoro (Tonari no Totoro) (1988)</td>\n",
       "      <td>Reservoir Dogs (1992)</td>\n",
       "      <td>Grifters, The (1990)</td>\n",
       "      <td>Rashomon (RashÃ´mon) (1950)</td>\n",
       "      <td>Spider-Man (2002)</td>\n",
       "      <td>Belladonna of Sadness (1973)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>My Neighbor Totoro (Tonari no Totoro) (1988)</td>\n",
       "      <td>Requiem for a Dream (2000)</td>\n",
       "      <td>Pokemon 4 Ever (a.k.a. PokÃ©mon 4: The Movie) ...</td>\n",
       "      <td>Minority Report (2002)</td>\n",
       "      <td>I Am a Fugitive from a Chain Gang (1932)</td>\n",
       "      <td>Chinatown (1974)</td>\n",
       "      <td>Blade Runner (1982)</td>\n",
       "      <td>Man Who Would Be King, The (1975)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Pokemon 4 Ever (a.k.a. PokÃ©mon 4: The Movie) ...</td>\n",
       "      <td>Shakespeare in Love (1998)</td>\n",
       "      <td>Ice Age (2002)</td>\n",
       "      <td>Dead Man Walking (1995)</td>\n",
       "      <td>Bitter Moon (1992)</td>\n",
       "      <td>Mission: Impossible (1996)</td>\n",
       "      <td>Star Wars: Episode I - The Phantom Menace (1999)</td>\n",
       "      <td>Conversation, The (1974)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            Animation  \\\n",
       "0                                        Shrek (2001)   \n",
       "1                                  Toy Story 2 (1999)   \n",
       "2   Spirited Away (Sen to Chihiro no kamikakushi) ...   \n",
       "3                               Monsters, Inc. (2001)   \n",
       "4                                 Finding Nemo (2003)   \n",
       "5                               Lion King, The (1994)   \n",
       "6                             Incredibles, The (2004)   \n",
       "7   Wallace & Gromit: The Best of Aardman Animatio...   \n",
       "8                                      Shrek 2 (2004)   \n",
       "9                                    Toy Story (1995)   \n",
       "10  Howl's Moving Castle (Hauru no ugoku shiro) (2...   \n",
       "11                        Beauty and the Beast (1991)   \n",
       "12           Princess Mononoke (Mononoke-hime) (1997)   \n",
       "13                                          Up (2009)   \n",
       "14             Wallace & Gromit: A Close Shave (1995)   \n",
       "15    Grand Day Out with Wallace and Gromit, A (1989)   \n",
       "16                             Iron Giant, The (1999)   \n",
       "17                    Who Framed Roger Rabbit? (1988)   \n",
       "18                                     WALLÂ·E (2008)   \n",
       "19  NausicaÃ¤ of the Valley of the Wind (Kaze no t...   \n",
       "20                               Bug's Life, A (1998)   \n",
       "21                                     Aladdin (1992)   \n",
       "22        Wallace & Gromit: The Wrong Trousers (1993)   \n",
       "23       My Neighbor Totoro (Tonari no Totoro) (1988)   \n",
       "24  Pokemon 4 Ever (a.k.a. PokÃ©mon 4: The Movie) ...   \n",
       "\n",
       "                                                Drama  \\\n",
       "0                            Good Will Hunting (1997)   \n",
       "1        Eternal Sunshine of the Spotless Mind (2004)   \n",
       "2   Lord of the Rings: The Return of the King, The...   \n",
       "3                             Schindler's List (1993)   \n",
       "4   LÃ©on: The Professional (a.k.a. The Profession...   \n",
       "5                            Beautiful Mind, A (2001)   \n",
       "6         Life Is Beautiful (La Vita Ã¨ bella) (1997)   \n",
       "7                                 Forrest Gump (1994)   \n",
       "8                               Godfather, The (1972)   \n",
       "9                               Lion King, The (1994)   \n",
       "10                             American Beauty (1999)   \n",
       "11                       2001: A Space Odyssey (1968)   \n",
       "12                                   Apollo 13 (1995)   \n",
       "13                                Pulp Fiction (1994)   \n",
       "14                City of God (Cidade de Deus) (2002)   \n",
       "15                                Donnie Darko (2001)   \n",
       "16  Crouching Tiger, Hidden Dragon (Wo hu cang lon...   \n",
       "17                                       Fargo (1996)   \n",
       "18                             Erin Brockovich (2000)   \n",
       "19                                   Cast Away (2000)   \n",
       "20                       To Kill a Mockingbird (1962)   \n",
       "21                         Saving Private Ryan (1998)   \n",
       "22                                    Big Fish (2003)   \n",
       "23                         Requiem for a Dream (2000)   \n",
       "24                         Shakespeare in Love (1998)   \n",
       "\n",
       "                                             Children  \\\n",
       "0                                        Shrek (2001)   \n",
       "1                                  Toy Story 2 (1999)   \n",
       "2                               Monsters, Inc. (2001)   \n",
       "3                                 Finding Nemo (2003)   \n",
       "4                               Lion King, The (1994)   \n",
       "5                             Incredibles, The (2004)   \n",
       "6                                      Shrek 2 (2004)   \n",
       "7                                    Toy Story (1995)   \n",
       "8                         Beauty and the Beast (1991)   \n",
       "9                                           Up (2009)   \n",
       "10  Harry Potter and the Sorcerer's Stone (a.k.a. ...   \n",
       "11             Wallace & Gromit: A Close Shave (1995)   \n",
       "12  Chronicles of Narnia: The Lion, the Witch and ...   \n",
       "13    Grand Day Out with Wallace and Gromit, A (1989)   \n",
       "14                             Iron Giant, The (1999)   \n",
       "15                                        Babe (1995)   \n",
       "16                    Who Framed Roger Rabbit? (1988)   \n",
       "17                                     WALLÂ·E (2008)   \n",
       "18           Charlie and the Chocolate Factory (2005)   \n",
       "19                               Bug's Life, A (1998)   \n",
       "20                                     Aladdin (1992)   \n",
       "21        Wallace & Gromit: The Wrong Trousers (1993)   \n",
       "22       My Neighbor Totoro (Tonari no Totoro) (1988)   \n",
       "23  Pokemon 4 Ever (a.k.a. PokÃ©mon 4: The Movie) ...   \n",
       "24                                     Ice Age (2002)   \n",
       "\n",
       "                                                Crime  \\\n",
       "0   LÃ©on: The Professional (a.k.a. The Profession...   \n",
       "1                               Godfather, The (1972)   \n",
       "2                                 Pulp Fiction (1994)   \n",
       "3                 City of God (Cidade de Deus) (2002)   \n",
       "4                                        Fargo (1996)   \n",
       "5                   O Brother, Where Art Thou? (2000)   \n",
       "6                              Green Mile, The (1999)   \n",
       "7                                  Taxi Driver (1976)   \n",
       "8                          Catch Me If You Can (2002)   \n",
       "9                                Trainspotting (1996)   \n",
       "10                   Shawshank Redemption, The (1994)   \n",
       "11                               Departed, The (2006)   \n",
       "12                                       Crash (2004)   \n",
       "13                         Clockwork Orange, A (1971)   \n",
       "14                   Silence of the Lambs, The (1991)   \n",
       "15                            Dark Knight, The (2008)   \n",
       "16                                  Fight Club (1999)   \n",
       "17                          American History X (1998)   \n",
       "18                     Godfather: Part II, The (1974)   \n",
       "19                    Who Framed Roger Rabbit? (1988)   \n",
       "20                                  Goodfellas (1990)   \n",
       "21        Wallace & Gromit: The Wrong Trousers (1993)   \n",
       "22                              Reservoir Dogs (1992)   \n",
       "23                             Minority Report (2002)   \n",
       "24                            Dead Man Walking (1995)   \n",
       "\n",
       "                                        Film-Noir  \\\n",
       "0                         Mulholland Drive (2001)   \n",
       "1                                Chinatown (1974)   \n",
       "2                        L.A. Confidential (1997)   \n",
       "3                                Notorious (1946)   \n",
       "4                                 Sin City (2005)   \n",
       "5                     Strangers on a Train (1951)   \n",
       "6                                Dark City (1998)   \n",
       "7   Sunset Blvd. (a.k.a. Sunset Boulevard) (1950)   \n",
       "8                        Miller's Crossing (1990)   \n",
       "9                                Key Largo (1948)   \n",
       "10                        Double Indemnity (1944)   \n",
       "11                   While the City Sleeps (1956)   \n",
       "12                Night of the Hunter, The (1955)   \n",
       "13                            Lost Highway (1997)   \n",
       "14                           Touch of Evil (1958)   \n",
       "15                    Stray Dog (Nora inu) (1949)   \n",
       "16                                       M (1931)   \n",
       "17                            Dark Passage (1947)   \n",
       "18                      Call Northside 777 (1948)   \n",
       "19                             High Sierra (1941)   \n",
       "20                                   Gilda (1946)   \n",
       "21                            Johnny Eager (1942)   \n",
       "22                           Grifters, The (1990)   \n",
       "23       I Am a Fugitive from a Chain Gang (1932)   \n",
       "24                             Bitter Moon (1992)   \n",
       "\n",
       "                                              Mystery  \\\n",
       "0                                 Donnie Darko (2001)   \n",
       "1                             Sixth Sense, The (1999)   \n",
       "2                                    Game, The (1997)   \n",
       "3           Twelve Monkeys (a.k.a. 12 Monkeys) (1995)   \n",
       "4   City of Lost Children, The (CitÃ© des enfants ...   \n",
       "5                                      Vertigo (1958)   \n",
       "6                     Who Framed Roger Rabbit? (1988)   \n",
       "7                                      Memento (2000)   \n",
       "8                               Reservoir Dogs (1992)   \n",
       "9                              Minority Report (2002)   \n",
       "10                        Seven (a.k.a. Se7en) (1995)   \n",
       "11                               The Machinist (2004)   \n",
       "12                         Usual Suspects, The (1995)   \n",
       "13                          North by Northwest (1959)   \n",
       "14                               Prestige, The (2006)   \n",
       "15                                   Inception (2010)   \n",
       "16                                Citizen Kane (1941)   \n",
       "17                                 Rear Window (1954)   \n",
       "18                              Eyes Wide Shut (1999)   \n",
       "19                                Gosford Park (2001)   \n",
       "20                                Mystic River (2003)   \n",
       "21                            Mulholland Drive (2001)   \n",
       "22                        Rashomon (RashÃ´mon) (1950)   \n",
       "23                                   Chinatown (1974)   \n",
       "24                         Mission: Impossible (1996)   \n",
       "\n",
       "                                               Sci-Fi  \\\n",
       "0        Eternal Sunshine of the Spotless Mind (2004)   \n",
       "1                        2001: A Space Odyssey (1968)   \n",
       "2                                 Donnie Darko (2001)   \n",
       "3                         Matrix Reloaded, The (2003)   \n",
       "4                                        X-Men (2000)   \n",
       "5   Star Wars: Episode VI - Return of the Jedi (1983)   \n",
       "6                      Matrix Revolutions, The (2003)   \n",
       "7                          Clockwork Orange, A (1971)   \n",
       "8                Independence Day (a.k.a. ID4) (1996)   \n",
       "9                           Back to the Future (1985)   \n",
       "10                              V for Vendetta (2006)   \n",
       "11          Twelve Monkeys (a.k.a. 12 Monkeys) (1995)   \n",
       "12          Star Wars: Episode IV - A New Hope (1977)   \n",
       "13  City of Lost Children, The (CitÃ© des enfants ...   \n",
       "14    Grand Day Out with Wallace and Gromit, A (1989)   \n",
       "15                             Iron Giant, The (1999)   \n",
       "16                                     WALLÂ·E (2008)   \n",
       "17                                Spider-Man 2 (2004)   \n",
       "18  NausicaÃ¤ of the Valley of the Wind (Kaze no t...   \n",
       "19                                     Contact (1997)   \n",
       "20                            Truman Show, The (1998)   \n",
       "21                            X2: X-Men United (2003)   \n",
       "22                                  Spider-Man (2002)   \n",
       "23                                Blade Runner (1982)   \n",
       "24   Star Wars: Episode I - The Phantom Menace (1999)   \n",
       "\n",
       "                                                1970s  \n",
       "0                               Godfather, The (1972)  \n",
       "1              One Flew Over the Cuckoo's Nest (1975)  \n",
       "2                                  Taxi Driver (1976)  \n",
       "3                          Clockwork Orange, A (1971)  \n",
       "4              Monty Python and the Holy Grail (1975)  \n",
       "5           Star Wars: Episode IV - A New Hope (1977)  \n",
       "6                      Godfather: Part II, The (1974)  \n",
       "7                               Apocalypse Now (1979)  \n",
       "8                                   Robin Hood (1973)  \n",
       "9          Willy Wonka & the Chocolate Factory (1971)  \n",
       "10                           American Graffiti (1973)  \n",
       "11                                  Sting, The (1973)  \n",
       "12                                       Alien (1979)  \n",
       "13                Monty Python's Life of Brian (1979)  \n",
       "14                                 Deliverance (1972)  \n",
       "15                          Young Frankenstein (1974)  \n",
       "16  Aguirre: The Wrath of God (Aguirre, der Zorn G...  \n",
       "17                                   Chinatown (1974)  \n",
       "18                       M*A*S*H (a.k.a. MASH) (1970)  \n",
       "19          Close Encounters of the Third Kind (1977)  \n",
       "20                            Deer Hunter, The (1978)  \n",
       "21                                    Badlands (1973)  \n",
       "22                       Belladonna of Sadness (1973)  \n",
       "23                  Man Who Would Be King, The (1975)  \n",
       "24                           Conversation, The (1974)  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recommendations_df_shelves = pd.DataFrame()\n",
    "for filter_arn in meta_filter_arns:\n",
    "    recommendations_df_shelves = get_new_recommendations_df_by_filter(recommendations_df_shelves, user, filter_arn)\n",
    "for filter_arn in decade_filter_arns:\n",
    "    recommendations_df_shelves = get_new_recommendations_df_by_filter(recommendations_df_shelves, user, filter_arn)\n",
    "\n",
    "recommendations_df_shelves"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next topic is real-time events. Personalize has the ability to listen to events from your application in order to update the recommendations shown to the user. This is especially useful in media workloads, like video-on-demand, where a customer's intent may differ based on if they are watching with their children or on their own.\n",
    "\n",
    "Additionally the events that are recorded via this system are stored until a delete call from you is issued, and they are used as historical data alongside the other interaction data you provided when you train your next models.\n",
    "\n",
    "#### Real time events\n",
    "\n",
    "Start by creating an event tracker that is attached to the campaign."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "arn:aws:personalize:us-east-1:835319576252:event-tracker/1e2b508b\n",
      "de689434-35d5-489a-9812-249066937d6a\n"
     ]
    }
   ],
   "source": [
    "response = personalize.create_event_tracker(\n",
    "    name='MovieTracker',\n",
    "    datasetGroupArn=dataset_group_arn\n",
    ")\n",
    "print(response['eventTrackerArn'])\n",
    "print(response['trackingId'])\n",
    "TRACKING_ID = response['trackingId']\n",
    "event_tracker_arn = response['eventTrackerArn']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will create some code that simulates a user interacting with a particular item. After running this code, you will get recommendations that differ from the results above.\n",
    "\n",
    "We start by creating some methods for the simulation of real time events."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "session_dict = {}\n",
    "\n",
    "def send_movie_click(USER_ID, ITEM_ID, EVENT_TYPE):\n",
    "    \"\"\"\n",
    "    Simulates a click as an envent\n",
    "    to send an event to Amazon Personalize's Event Tracker\n",
    "    \"\"\"\n",
    "    # Configure Session\n",
    "    try:\n",
    "        session_ID = session_dict[str(USER_ID)]\n",
    "    except:\n",
    "        session_dict[str(USER_ID)] = str(uuid.uuid1())\n",
    "        session_ID = session_dict[str(USER_ID)]\n",
    "        \n",
    "    # Configure Properties:\n",
    "    event = {\n",
    "    \"itemId\": str(ITEM_ID),\n",
    "    }\n",
    "    event_json = json.dumps(event)\n",
    "        \n",
    "    # Make Call\n",
    "    \n",
    "    personalize_events.put_events(\n",
    "    trackingId = TRACKING_ID,\n",
    "    userId= str(USER_ID),\n",
    "    sessionId = session_ID,\n",
    "    eventList = [{\n",
    "        'sentAt': int(time.time()),\n",
    "        'eventType': str(EVENT_TYPE),\n",
    "        'properties': event_json\n",
    "        }]\n",
    "    )\n",
    "\n",
    "def get_new_recommendations_df_users_real_time(recommendations_df, user_id, item_id, event_type):\n",
    "    # Get the artist name (header of column)\n",
    "    movie_name = get_movie_by_id(item_id)\n",
    "    # Interact with different movies\n",
    "    print('sending event ' + event_type + ' for ' + get_movie_by_id(item_id))\n",
    "    send_movie_click(USER_ID=user_id, ITEM_ID=item_id, EVENT_TYPE=event_type)\n",
    "    # Get the recommendations (note you should have a base recommendation DF created before)\n",
    "    get_recommendations_response = personalize_runtime.get_recommendations(\n",
    "        campaignArn = userpersonalization_campaign_arn,\n",
    "        userId = str(user_id),\n",
    "    )\n",
    "    # Build a new dataframe of recommendations\n",
    "    item_list = get_recommendations_response['itemList']\n",
    "    recommendation_list = []\n",
    "    for item in item_list:\n",
    "        artist = get_movie_by_id(item['itemId'])\n",
    "        recommendation_list.append(artist)\n",
    "    new_rec_DF = pd.DataFrame(recommendation_list, columns = [movie_name])\n",
    "    # Add this dataframe to the old one\n",
    "    #recommendations_df = recommendations_df.join(new_rec_DF)\n",
    "    recommendations_df = pd.concat([recommendations_df, new_rec_DF], axis=1)\n",
    "    return recommendations_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point, we haven't generated any real-time events yet; we have only set up the code. To compare the recommendations before and after the real-time events, let's pick one user and generate the original recommendations for them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>396</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Shrek (2001)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Amelie (Fabuleux destin d'AmÃ©lie Poulain, Le)...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lord of the Rings: The Two Towers, The (2002)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Toy Story 2 (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Good Will Hunting (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Eternal Sunshine of the Spotless Mind (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Spirited Away (Sen to Chihiro no kamikakushi) ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Lord of the Rings: The Return of the King, The...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Schindler's List (1993)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>LÃ©on: The Professional (a.k.a. The Profession...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Beautiful Mind, A (2001)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Monsters, Inc. (2001)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Life Is Beautiful (La Vita Ã¨ bella) (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Forrest Gump (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Godfather, The (1972)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Finding Nemo (2003)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Lion King, The (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>American Beauty (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Lord of the Rings: The Fellowship of the Ring,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>2001: A Space Odyssey (1968)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Pirates of the Caribbean: The Curse of the Bla...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Apollo 13 (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Pulp Fiction (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Harry Potter and the Prisoner of Azkaban (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>City of God (Cidade de Deus) (2002)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                  396\n",
       "0                                        Shrek (2001)\n",
       "1   Amelie (Fabuleux destin d'AmÃ©lie Poulain, Le)...\n",
       "2       Lord of the Rings: The Two Towers, The (2002)\n",
       "3                                  Toy Story 2 (1999)\n",
       "4                            Good Will Hunting (1997)\n",
       "5        Eternal Sunshine of the Spotless Mind (2004)\n",
       "6   Spirited Away (Sen to Chihiro no kamikakushi) ...\n",
       "7   Lord of the Rings: The Return of the King, The...\n",
       "8                             Schindler's List (1993)\n",
       "9   LÃ©on: The Professional (a.k.a. The Profession...\n",
       "10                           Beautiful Mind, A (2001)\n",
       "11                              Monsters, Inc. (2001)\n",
       "12        Life Is Beautiful (La Vita Ã¨ bella) (1997)\n",
       "13                                Forrest Gump (1994)\n",
       "14                              Godfather, The (1972)\n",
       "15                                Finding Nemo (2003)\n",
       "16                              Lion King, The (1994)\n",
       "17                             American Beauty (1999)\n",
       "18  Lord of the Rings: The Fellowship of the Ring,...\n",
       "19                       2001: A Space Odyssey (1968)\n",
       "20  Pirates of the Caribbean: The Curse of the Bla...\n",
       "21                                   Apollo 13 (1995)\n",
       "22                                Pulp Fiction (1994)\n",
       "23    Harry Potter and the Prisoner of Azkaban (2004)\n",
       "24                City of God (Cidade de Deus) (2002)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# First pick a user\n",
    "user_id = user\n",
    "\n",
    "# Get recommendations for the user\n",
    "get_recommendations_response = personalize_runtime.get_recommendations(\n",
    "        campaignArn = userpersonalization_campaign_arn,\n",
    "        userId = str(user_id),\n",
    "    )\n",
    "\n",
    "# Build a new dataframe for the recommendations\n",
    "item_list = get_recommendations_response['itemList']\n",
    "recommendation_list = []\n",
    "for item in item_list:\n",
    "    artist = get_movie_by_id(item['itemId'])\n",
    "    recommendation_list.append(artist)\n",
    "user_recommendations_df = pd.DataFrame(recommendation_list, columns = [user_id])\n",
    "user_recommendations_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ok, so now we have a list of recommendations for this user before we have applied any real-time events. Now let's pick 3 random artists which we will simulate our user interacting with, and then see how this changes the recommendations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Next generate 3 random movies\n",
    "movies = items_df.sample(3).index.tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sending event click for Pollyanna (1960)\n",
      "sending event click for Somers Town (2008)\n",
      "sending event click for Guest from the Future (Gostya iz buduschego) (1985)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>396</th>\n",
       "      <th>Pollyanna (1960)</th>\n",
       "      <th>Somers Town (2008)</th>\n",
       "      <th>Guest from the Future (Gostya iz buduschego) (1985)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Shrek (2001)</td>\n",
       "      <td>Shrek (2001)</td>\n",
       "      <td>American Graffiti (1973)</td>\n",
       "      <td>Shrek (2001)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Amelie (Fabuleux destin d'AmÃ©lie Poulain, Le)...</td>\n",
       "      <td>Amelie (Fabuleux destin d'AmÃ©lie Poulain, Le)...</td>\n",
       "      <td>Graduate, The (1967)</td>\n",
       "      <td>Amelie (Fabuleux destin d'AmÃ©lie Poulain, Le)...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lord of the Rings: The Two Towers, The (2002)</td>\n",
       "      <td>Lord of the Rings: The Two Towers, The (2002)</td>\n",
       "      <td>Miracle on 34th Street (1947)</td>\n",
       "      <td>Lord of the Rings: The Two Towers, The (2002)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Toy Story 2 (1999)</td>\n",
       "      <td>Toy Story 2 (1999)</td>\n",
       "      <td>Producers, The (1968)</td>\n",
       "      <td>Toy Story 2 (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Good Will Hunting (1997)</td>\n",
       "      <td>Good Will Hunting (1997)</td>\n",
       "      <td>Willy Wonka &amp; the Chocolate Factory (1971)</td>\n",
       "      <td>Good Will Hunting (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Eternal Sunshine of the Spotless Mind (2004)</td>\n",
       "      <td>Eternal Sunshine of the Spotless Mind (2004)</td>\n",
       "      <td>Charlie Brown Christmas, A (1965)</td>\n",
       "      <td>Eternal Sunshine of the Spotless Mind (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Spirited Away (Sen to Chihiro no kamikakushi) ...</td>\n",
       "      <td>Spirited Away (Sen to Chihiro no kamikakushi) ...</td>\n",
       "      <td>Monty Python's And Now for Something Completel...</td>\n",
       "      <td>Spirited Away (Sen to Chihiro no kamikakushi) ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Lord of the Rings: The Return of the King, The...</td>\n",
       "      <td>Lord of the Rings: The Return of the King, The...</td>\n",
       "      <td>Apartment, The (1960)</td>\n",
       "      <td>Lord of the Rings: The Return of the King, The...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Schindler's List (1993)</td>\n",
       "      <td>Schindler's List (1993)</td>\n",
       "      <td>Sleeper (1973)</td>\n",
       "      <td>Schindler's List (1993)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>LÃ©on: The Professional (a.k.a. The Profession...</td>\n",
       "      <td>LÃ©on: The Professional (a.k.a. The Profession...</td>\n",
       "      <td>Philadelphia Story, The (1940)</td>\n",
       "      <td>LÃ©on: The Professional (a.k.a. The Profession...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Beautiful Mind, A (2001)</td>\n",
       "      <td>Beautiful Mind, A (2001)</td>\n",
       "      <td>To Kill a Mockingbird (1962)</td>\n",
       "      <td>Beautiful Mind, A (2001)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Monsters, Inc. (2001)</td>\n",
       "      <td>Monsters, Inc. (2001)</td>\n",
       "      <td>Mary Poppins (1964)</td>\n",
       "      <td>Monsters, Inc. (2001)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Life Is Beautiful (La Vita Ã¨ bella) (1997)</td>\n",
       "      <td>Life Is Beautiful (La Vita Ã¨ bella) (1997)</td>\n",
       "      <td>Follow Me, Boys! (1966)</td>\n",
       "      <td>Life Is Beautiful (La Vita Ã¨ bella) (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Forrest Gump (1994)</td>\n",
       "      <td>Forrest Gump (1994)</td>\n",
       "      <td>Jungle Book, The (1967)</td>\n",
       "      <td>Forrest Gump (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Godfather, The (1972)</td>\n",
       "      <td>Godfather, The (1972)</td>\n",
       "      <td>Pollyanna (1960)</td>\n",
       "      <td>Godfather, The (1972)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Finding Nemo (2003)</td>\n",
       "      <td>Finding Nemo (2003)</td>\n",
       "      <td>Modern Times (1936)</td>\n",
       "      <td>Finding Nemo (2003)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Lion King, The (1994)</td>\n",
       "      <td>Lion King, The (1994)</td>\n",
       "      <td>It's the Great Pumpkin, Charlie Brown (1966)</td>\n",
       "      <td>Lion King, The (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>American Beauty (1999)</td>\n",
       "      <td>American Beauty (1999)</td>\n",
       "      <td>Father of the Bride (1950)</td>\n",
       "      <td>American Beauty (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Lord of the Rings: The Fellowship of the Ring,...</td>\n",
       "      <td>Lord of the Rings: The Fellowship of the Ring,...</td>\n",
       "      <td>Diary of a Chambermaid (Journal d'une femme de...</td>\n",
       "      <td>Lord of the Rings: The Fellowship of the Ring,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>2001: A Space Odyssey (1968)</td>\n",
       "      <td>2001: A Space Odyssey (1968)</td>\n",
       "      <td>Harold and Maude (1971)</td>\n",
       "      <td>2001: A Space Odyssey (1968)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Pirates of the Caribbean: The Curse of the Bla...</td>\n",
       "      <td>Pirates of the Caribbean: The Curse of the Bla...</td>\n",
       "      <td>Pygmalion (1938)</td>\n",
       "      <td>Pirates of the Caribbean: The Curse of the Bla...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Apollo 13 (1995)</td>\n",
       "      <td>Apollo 13 (1995)</td>\n",
       "      <td>Day the Earth Stood Still, The (1951)</td>\n",
       "      <td>Apollo 13 (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Pulp Fiction (1994)</td>\n",
       "      <td>Pulp Fiction (1994)</td>\n",
       "      <td>Absent-Minded Professor, The (1961)</td>\n",
       "      <td>Pulp Fiction (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Harry Potter and the Prisoner of Azkaban (2004)</td>\n",
       "      <td>Harry Potter and the Prisoner of Azkaban (2004)</td>\n",
       "      <td>Casablanca (1942)</td>\n",
       "      <td>Harry Potter and the Prisoner of Azkaban (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>City of God (Cidade de Deus) (2002)</td>\n",
       "      <td>City of God (Cidade de Deus) (2002)</td>\n",
       "      <td>Dr. Strangelove or: How I Learned to Stop Worr...</td>\n",
       "      <td>City of God (Cidade de Deus) (2002)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                  396  \\\n",
       "0                                        Shrek (2001)   \n",
       "1   Amelie (Fabuleux destin d'AmÃ©lie Poulain, Le)...   \n",
       "2       Lord of the Rings: The Two Towers, The (2002)   \n",
       "3                                  Toy Story 2 (1999)   \n",
       "4                            Good Will Hunting (1997)   \n",
       "5        Eternal Sunshine of the Spotless Mind (2004)   \n",
       "6   Spirited Away (Sen to Chihiro no kamikakushi) ...   \n",
       "7   Lord of the Rings: The Return of the King, The...   \n",
       "8                             Schindler's List (1993)   \n",
       "9   LÃ©on: The Professional (a.k.a. The Profession...   \n",
       "10                           Beautiful Mind, A (2001)   \n",
       "11                              Monsters, Inc. (2001)   \n",
       "12        Life Is Beautiful (La Vita Ã¨ bella) (1997)   \n",
       "13                                Forrest Gump (1994)   \n",
       "14                              Godfather, The (1972)   \n",
       "15                                Finding Nemo (2003)   \n",
       "16                              Lion King, The (1994)   \n",
       "17                             American Beauty (1999)   \n",
       "18  Lord of the Rings: The Fellowship of the Ring,...   \n",
       "19                       2001: A Space Odyssey (1968)   \n",
       "20  Pirates of the Caribbean: The Curse of the Bla...   \n",
       "21                                   Apollo 13 (1995)   \n",
       "22                                Pulp Fiction (1994)   \n",
       "23    Harry Potter and the Prisoner of Azkaban (2004)   \n",
       "24                City of God (Cidade de Deus) (2002)   \n",
       "\n",
       "                                     Pollyanna (1960)  \\\n",
       "0                                        Shrek (2001)   \n",
       "1   Amelie (Fabuleux destin d'AmÃ©lie Poulain, Le)...   \n",
       "2       Lord of the Rings: The Two Towers, The (2002)   \n",
       "3                                  Toy Story 2 (1999)   \n",
       "4                            Good Will Hunting (1997)   \n",
       "5        Eternal Sunshine of the Spotless Mind (2004)   \n",
       "6   Spirited Away (Sen to Chihiro no kamikakushi) ...   \n",
       "7   Lord of the Rings: The Return of the King, The...   \n",
       "8                             Schindler's List (1993)   \n",
       "9   LÃ©on: The Professional (a.k.a. The Profession...   \n",
       "10                           Beautiful Mind, A (2001)   \n",
       "11                              Monsters, Inc. (2001)   \n",
       "12        Life Is Beautiful (La Vita Ã¨ bella) (1997)   \n",
       "13                                Forrest Gump (1994)   \n",
       "14                              Godfather, The (1972)   \n",
       "15                                Finding Nemo (2003)   \n",
       "16                              Lion King, The (1994)   \n",
       "17                             American Beauty (1999)   \n",
       "18  Lord of the Rings: The Fellowship of the Ring,...   \n",
       "19                       2001: A Space Odyssey (1968)   \n",
       "20  Pirates of the Caribbean: The Curse of the Bla...   \n",
       "21                                   Apollo 13 (1995)   \n",
       "22                                Pulp Fiction (1994)   \n",
       "23    Harry Potter and the Prisoner of Azkaban (2004)   \n",
       "24                City of God (Cidade de Deus) (2002)   \n",
       "\n",
       "                                   Somers Town (2008)  \\\n",
       "0                            American Graffiti (1973)   \n",
       "1                                Graduate, The (1967)   \n",
       "2                       Miracle on 34th Street (1947)   \n",
       "3                               Producers, The (1968)   \n",
       "4          Willy Wonka & the Chocolate Factory (1971)   \n",
       "5                   Charlie Brown Christmas, A (1965)   \n",
       "6   Monty Python's And Now for Something Completel...   \n",
       "7                               Apartment, The (1960)   \n",
       "8                                      Sleeper (1973)   \n",
       "9                      Philadelphia Story, The (1940)   \n",
       "10                       To Kill a Mockingbird (1962)   \n",
       "11                                Mary Poppins (1964)   \n",
       "12                            Follow Me, Boys! (1966)   \n",
       "13                            Jungle Book, The (1967)   \n",
       "14                                   Pollyanna (1960)   \n",
       "15                                Modern Times (1936)   \n",
       "16       It's the Great Pumpkin, Charlie Brown (1966)   \n",
       "17                         Father of the Bride (1950)   \n",
       "18  Diary of a Chambermaid (Journal d'une femme de...   \n",
       "19                            Harold and Maude (1971)   \n",
       "20                                   Pygmalion (1938)   \n",
       "21              Day the Earth Stood Still, The (1951)   \n",
       "22                Absent-Minded Professor, The (1961)   \n",
       "23                                  Casablanca (1942)   \n",
       "24  Dr. Strangelove or: How I Learned to Stop Worr...   \n",
       "\n",
       "   Guest from the Future (Gostya iz buduschego) (1985)  \n",
       "0                                        Shrek (2001)   \n",
       "1   Amelie (Fabuleux destin d'AmÃ©lie Poulain, Le)...   \n",
       "2       Lord of the Rings: The Two Towers, The (2002)   \n",
       "3                                  Toy Story 2 (1999)   \n",
       "4                            Good Will Hunting (1997)   \n",
       "5        Eternal Sunshine of the Spotless Mind (2004)   \n",
       "6   Spirited Away (Sen to Chihiro no kamikakushi) ...   \n",
       "7   Lord of the Rings: The Return of the King, The...   \n",
       "8                             Schindler's List (1993)   \n",
       "9   LÃ©on: The Professional (a.k.a. The Profession...   \n",
       "10                           Beautiful Mind, A (2001)   \n",
       "11                              Monsters, Inc. (2001)   \n",
       "12        Life Is Beautiful (La Vita Ã¨ bella) (1997)   \n",
       "13                                Forrest Gump (1994)   \n",
       "14                              Godfather, The (1972)   \n",
       "15                                Finding Nemo (2003)   \n",
       "16                              Lion King, The (1994)   \n",
       "17                             American Beauty (1999)   \n",
       "18  Lord of the Rings: The Fellowship of the Ring,...   \n",
       "19                       2001: A Space Odyssey (1968)   \n",
       "20  Pirates of the Caribbean: The Curse of the Bla...   \n",
       "21                                   Apollo 13 (1995)   \n",
       "22                                Pulp Fiction (1994)   \n",
       "23    Harry Potter and the Prisoner of Azkaban (2004)   \n",
       "24                City of God (Cidade de Deus) (2002)   "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Note this will take about 15 seconds to complete due to the sleeps\n",
    "for movie in movies:\n",
    "    time.sleep(5)\n",
    "    user_recommendations_df = get_new_recommendations_df_users_real_time(user_recommendations_df, user_id, movie,'click')\n",
    "user_recommendations_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the cell above, the first column after the index is the user's default recommendations from HRNN, and each column after that has a header of the artist that they interacted with via a real time event, and the recommendations after this event occurred. \n",
    "\n",
    "The behavior may not shift very much; this is due to the relatively limited nature of this dataset and effect of a few random clicks. If you wanted to better understand this, try simulating clicking more movies, and you should see a more pronounced impact."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now lets look at the event filters, which allow you to filter items based on the interaction data. For this dataset, it could be click or watch based on the data we imported, but could be based on whatever interaction schema you design (click, rate, like, watch, purchase etc.) For VOD shelves you could move a title from \"Top picks for you\" to a \"Watch again\", the watch again recommendations will be based on the users current interactions, but only recommend titles that have already been watched.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>watched</th>\n",
       "      <th>unwatched</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LÃ©on: The Professional (a.k.a. The Profession...</td>\n",
       "      <td>Inglourious Basterds (2009)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Up (2009)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Dark Knight, The (2008)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Eternal Sunshine of the Spotless Mind (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Forrest Gump (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Little Miss Sunshine (2006)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NaN</td>\n",
       "      <td>WALLÂ·E (2008)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Thank You for Smoking (2006)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Children of Men (2006)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>NaN</td>\n",
       "      <td>V for Vendetta (2006)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Departed, The (2006)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Godfather: Part II, The (1974)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Borat: Cultural Learnings of America for Make ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Graduate, The (1967)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Saving Private Ryan (1998)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Star Trek (2009)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Watchmen (2009)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Iron Man (2008)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Godfather, The (1972)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Crouching Tiger, Hidden Dragon (Wo hu cang lon...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Schindler's List (1993)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Apocalypse Now (1979)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Ghostbusters (a.k.a. Ghost Busters) (1984)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>NaN</td>\n",
       "      <td>One Flew Over the Cuckoo's Nest (1975)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>NaN</td>\n",
       "      <td>300 (2007)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                              watched  \\\n",
       "0   LÃ©on: The Professional (a.k.a. The Profession...   \n",
       "1                                                 NaN   \n",
       "2                                                 NaN   \n",
       "3                                                 NaN   \n",
       "4                                                 NaN   \n",
       "5                                                 NaN   \n",
       "6                                                 NaN   \n",
       "7                                                 NaN   \n",
       "8                                                 NaN   \n",
       "9                                                 NaN   \n",
       "10                                                NaN   \n",
       "11                                                NaN   \n",
       "12                                                NaN   \n",
       "13                                                NaN   \n",
       "14                                                NaN   \n",
       "15                                                NaN   \n",
       "16                                                NaN   \n",
       "17                                                NaN   \n",
       "18                                                NaN   \n",
       "19                                                NaN   \n",
       "20                                                NaN   \n",
       "21                                                NaN   \n",
       "22                                                NaN   \n",
       "23                                                NaN   \n",
       "24                                                NaN   \n",
       "\n",
       "                                            unwatched  \n",
       "0                         Inglourious Basterds (2009)  \n",
       "1                                           Up (2009)  \n",
       "2                             Dark Knight, The (2008)  \n",
       "3        Eternal Sunshine of the Spotless Mind (2004)  \n",
       "4                                 Forrest Gump (1994)  \n",
       "5                         Little Miss Sunshine (2006)  \n",
       "6                                      WALLÂ·E (2008)  \n",
       "7                        Thank You for Smoking (2006)  \n",
       "8                              Children of Men (2006)  \n",
       "9                               V for Vendetta (2006)  \n",
       "10                               Departed, The (2006)  \n",
       "11                     Godfather: Part II, The (1974)  \n",
       "12  Borat: Cultural Learnings of America for Make ...  \n",
       "13                               Graduate, The (1967)  \n",
       "14                         Saving Private Ryan (1998)  \n",
       "15                                   Star Trek (2009)  \n",
       "16                                    Watchmen (2009)  \n",
       "17                                    Iron Man (2008)  \n",
       "18                              Godfather, The (1972)  \n",
       "19  Crouching Tiger, Hidden Dragon (Wo hu cang lon...  \n",
       "20                            Schindler's List (1993)  \n",
       "21                              Apocalypse Now (1979)  \n",
       "22         Ghostbusters (a.k.a. Ghost Busters) (1984)  \n",
       "23             One Flew Over the Cuckoo's Nest (1975)  \n",
       "24                                         300 (2007)  "
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recommendations_df_events = pd.DataFrame()\n",
    "for filter_arn in interaction_filter_arns:\n",
    "    recommendations_df_events = get_new_recommendations_df_by_filter(recommendations_df_events, user, filter_arn)\n",
    "    \n",
    "recommendations_df_events"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "now lets send a watch event in for 4 unwatched recommendations, which would simulate watching 4 movies. In a VOD application, you may choose to send in an event after they have watched a significant amount (over 75%) of a piece of content. Sending at 100% complete could miss people that stop short of the credits."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sending event watch for Inglourious Basterds (2009)\n",
      "sending event watch for Up (2009)\n",
      "sending event watch for Dark Knight, The (2008)\n",
      "sending event watch for Eternal Sunshine of the Spotless Mind (2004)\n"
     ]
    }
   ],
   "source": [
    " # Get the recommendations\n",
    "top_unwatched_recommendations_response = personalize_runtime.get_recommendations(\n",
    "    campaignArn = userpersonalization_campaign_arn,\n",
    "    userId = str(user_id),\n",
    "    filterArn = filter_arn,\n",
    "    numResults=4)\n",
    "item_list = top_unwatched_recommendations_response['itemList']\n",
    "for item in item_list:\n",
    "    print('sending event watch for ' + get_movie_by_id(item['itemId']))\n",
    "    send_movie_click(USER_ID=user_id, ITEM_ID=item['itemId'], EVENT_TYPE='watch')\n",
    "    time.sleep(10)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can look at the event filters to see the updated watched and unwatched recommendations "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>watched</th>\n",
       "      <th>unwatched</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Eternal Sunshine of the Spotless Mind (2004)</td>\n",
       "      <td>Shrek (2001)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Amelie (Fabuleux destin d'AmÃ©lie Poulain, Le)...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Lord of the Rings: The Two Towers, The (2002)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Good Will Hunting (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Eternal Sunshine of the Spotless Mind (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Spirited Away (Sen to Chihiro no kamikakushi) ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Lord of the Rings: The Return of the King, The...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Schindler's List (1993)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Beautiful Mind, A (2001)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Life Is Beautiful (La Vita Ã¨ bella) (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Forrest Gump (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Godfather, The (1972)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Lion King, The (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>NaN</td>\n",
       "      <td>American Beauty (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Lord of the Rings: The Fellowship of the Ring,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>NaN</td>\n",
       "      <td>2001: A Space Odyssey (1968)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Pirates of the Caribbean: The Curse of the Bla...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Apollo 13 (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Pulp Fiction (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Harry Potter and the Prisoner of Azkaban (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>NaN</td>\n",
       "      <td>City of God (Cidade de Deus) (2002)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Donnie Darko (2001)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Crouching Tiger, Hidden Dragon (Wo hu cang lon...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Fargo (1996)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Erin Brockovich (2000)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                         watched  \\\n",
       "0   Eternal Sunshine of the Spotless Mind (2004)   \n",
       "1                                            NaN   \n",
       "2                                            NaN   \n",
       "3                                            NaN   \n",
       "4                                            NaN   \n",
       "5                                            NaN   \n",
       "6                                            NaN   \n",
       "7                                            NaN   \n",
       "8                                            NaN   \n",
       "9                                            NaN   \n",
       "10                                           NaN   \n",
       "11                                           NaN   \n",
       "12                                           NaN   \n",
       "13                                           NaN   \n",
       "14                                           NaN   \n",
       "15                                           NaN   \n",
       "16                                           NaN   \n",
       "17                                           NaN   \n",
       "18                                           NaN   \n",
       "19                                           NaN   \n",
       "20                                           NaN   \n",
       "21                                           NaN   \n",
       "22                                           NaN   \n",
       "23                                           NaN   \n",
       "24                                           NaN   \n",
       "\n",
       "                                            unwatched  \n",
       "0                                        Shrek (2001)  \n",
       "1   Amelie (Fabuleux destin d'AmÃ©lie Poulain, Le)...  \n",
       "2       Lord of the Rings: The Two Towers, The (2002)  \n",
       "3                            Good Will Hunting (1997)  \n",
       "4        Eternal Sunshine of the Spotless Mind (2004)  \n",
       "5   Spirited Away (Sen to Chihiro no kamikakushi) ...  \n",
       "6   Lord of the Rings: The Return of the King, The...  \n",
       "7                             Schindler's List (1993)  \n",
       "8                            Beautiful Mind, A (2001)  \n",
       "9         Life Is Beautiful (La Vita Ã¨ bella) (1997)  \n",
       "10                                Forrest Gump (1994)  \n",
       "11                              Godfather, The (1972)  \n",
       "12                              Lion King, The (1994)  \n",
       "13                             American Beauty (1999)  \n",
       "14  Lord of the Rings: The Fellowship of the Ring,...  \n",
       "15                       2001: A Space Odyssey (1968)  \n",
       "16  Pirates of the Caribbean: The Curse of the Bla...  \n",
       "17                                   Apollo 13 (1995)  \n",
       "18                                Pulp Fiction (1994)  \n",
       "19    Harry Potter and the Prisoner of Azkaban (2004)  \n",
       "20                City of God (Cidade de Deus) (2002)  \n",
       "21                                Donnie Darko (2001)  \n",
       "22  Crouching Tiger, Hidden Dragon (Wo hu cang lon...  \n",
       "23                                       Fargo (1996)  \n",
       "24                             Erin Brockovich (2000)  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recommendations_df_events = pd.DataFrame()\n",
    "for filter_arn in interaction_filter_arns:\n",
    "    recommendations_df_events = get_new_recommendations_df_by_filter(recommendations_df_events, user, filter_arn)\n",
    "    \n",
    "recommendations_df_events"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Personalized Ranking\n",
    "\n",
    "The core use case for personalized ranking is to take a collection of items and to render them in priority or probable order of interest for a user. For a VOD application you want dynamically render a personalized shelf/rail/carousel based on some information (director, location, superhero franchise, movie time period etc). This may not be information that you have in your metadata, so a item metadata filter will not work, howeverr you may have this information within you system to generate the item list. \n",
    "\n",
    "To demonstrate this, we will use the same user from before and a random collection of items."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "rerank_user = user\n",
    "rerank_items = items_df.sample(25).index.tolist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now build a nice dataframe that shows the input data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Un-Ranked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Flintstones, The (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Mystery Train (1989)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Cashback (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Day Watch (Dnevnoy dozor) (2006)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Big (1988)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Fire and Ice (1983)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Life Stinks (1991)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Savages (2012)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Barefoot Contessa, The (1954)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Devil Dog: The Hound of Hell (1978)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Mickey's Once Upon a Christmas (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Tokyo! (2008)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>What Ever Happened to Baby Jane? (1962)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Meteor Man, The (1993)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Unknown White Male (2005)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Legend of 1900, The (a.k.a. The Legend of the ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Goodbye Lover (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Timecop (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>John Wick: Chapter Two (2017)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Masterminds (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Woodsman, The (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Flipped (2010)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Dragon Ball Z the Movie: The World's Strongest...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Perfect Murder, A (1998)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Comic-Con Episode IV: A Fan's Hope (2011)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            Un-Ranked\n",
       "0                             Flintstones, The (1994)\n",
       "1                                Mystery Train (1989)\n",
       "2                                     Cashback (2004)\n",
       "3                    Day Watch (Dnevnoy dozor) (2006)\n",
       "4                                          Big (1988)\n",
       "5                                 Fire and Ice (1983)\n",
       "6                                  Life Stinks (1991)\n",
       "7                                      Savages (2012)\n",
       "8                       Barefoot Contessa, The (1954)\n",
       "9                 Devil Dog: The Hound of Hell (1978)\n",
       "10              Mickey's Once Upon a Christmas (1999)\n",
       "11                                      Tokyo! (2008)\n",
       "12            What Ever Happened to Baby Jane? (1962)\n",
       "13                             Meteor Man, The (1993)\n",
       "14                          Unknown White Male (2005)\n",
       "15  Legend of 1900, The (a.k.a. The Legend of the ...\n",
       "16                               Goodbye Lover (1999)\n",
       "17                                     Timecop (1994)\n",
       "18                      John Wick: Chapter Two (2017)\n",
       "19                                 Masterminds (1997)\n",
       "20                               Woodsman, The (2004)\n",
       "21                                     Flipped (2010)\n",
       "22  Dragon Ball Z the Movie: The World's Strongest...\n",
       "23                           Perfect Murder, A (1998)\n",
       "24          Comic-Con Episode IV: A Fan's Hope (2011)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rerank_list = []\n",
    "for item in rerank_items:\n",
    "    movie = get_movie_by_id(item)\n",
    "    rerank_list.append(movie)\n",
    "rerank_df = pd.DataFrame(rerank_list, columns = ['Un-Ranked'])\n",
    "rerank_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then make the personalized ranking API call."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ResponseMetadata': {'RequestId': '90792154-2717-4e16-a595-ec1b2703e8a7',\n",
       "  'HTTPStatusCode': 200,\n",
       "  'HTTPHeaders': {'content-type': 'application/json',\n",
       "   'date': 'Sun, 08 Nov 2020 20:37:34 GMT',\n",
       "   'x-amzn-requestid': '90792154-2717-4e16-a595-ec1b2703e8a7',\n",
       "   'content-length': '1417',\n",
       "   'connection': 'keep-alive'},\n",
       "  'RetryAttempts': 0},\n",
       " 'personalizedRanking': [{'itemId': '355', 'score': 0.2689658},\n",
       "  {'itemId': '2797', 'score': 0.2234265},\n",
       "  {'itemId': '80586', 'score': 0.0924742},\n",
       "  {'itemId': '6203', 'score': 0.086949},\n",
       "  {'itemId': '379', 'score': 0.0706614},\n",
       "  {'itemId': '2691', 'score': 0.0666595},\n",
       "  {'itemId': '168248', 'score': 0.0433316},\n",
       "  {'itemId': '1892', 'score': 0.0300974},\n",
       "  {'itemId': '72692', 'score': 0.0287358},\n",
       "  {'itemId': '55207', 'score': 0.0266975},\n",
       "  {'itemId': '148982', 'score': 0.023446},\n",
       "  {'itemId': '3521', 'score': 0.0224833},\n",
       "  {'itemId': '3546', 'score': 0.0121619},\n",
       "  {'itemId': '4420', 'score': 0.0039101},\n",
       "  {'itemId': '52950'},\n",
       "  {'itemId': '5041'},\n",
       "  {'itemId': '95583'},\n",
       "  {'itemId': '63179'},\n",
       "  {'itemId': '6204'},\n",
       "  {'itemId': '44511'},\n",
       "  {'itemId': '2586'},\n",
       "  {'itemId': '8241'},\n",
       "  {'itemId': '30820'},\n",
       "  {'itemId': '95165'},\n",
       "  {'itemId': '94122'}],\n",
       " 'recommendationId': 'RID-1b823ef1-c3df-4682-8771-09899bfd262d'}"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Convert user to string:\n",
    "user_id = str(rerank_user)\n",
    "rerank_item_list = []\n",
    "for item in rerank_items:\n",
    "    rerank_item_list.append(str(item))\n",
    "    \n",
    "# Get recommended reranking\n",
    "get_recommendations_response_rerank = personalize_runtime.get_personalized_ranking(\n",
    "        campaignArn = rerank_campaign_arn,\n",
    "        userId = user_id,\n",
    "        inputList = rerank_item_list\n",
    ")\n",
    "\n",
    "get_recommendations_response_rerank"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now add the reranked items as a second column to the original dataframe, for a side-by-side comparison."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Un-Ranked</th>\n",
       "      <th>Re-Ranked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Flintstones, The (1994)</td>\n",
       "      <td>Flintstones, The (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Mystery Train (1989)</td>\n",
       "      <td>Big (1988)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Cashback (2004)</td>\n",
       "      <td>Flipped (2010)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Day Watch (Dnevnoy dozor) (2006)</td>\n",
       "      <td>Life Stinks (1991)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Big (1988)</td>\n",
       "      <td>Timecop (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Fire and Ice (1983)</td>\n",
       "      <td>Legend of 1900, The (a.k.a. The Legend of the ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Life Stinks (1991)</td>\n",
       "      <td>John Wick: Chapter Two (2017)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Savages (2012)</td>\n",
       "      <td>Perfect Murder, A (1998)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Barefoot Contessa, The (1954)</td>\n",
       "      <td>Mickey's Once Upon a Christmas (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Devil Dog: The Hound of Hell (1978)</td>\n",
       "      <td>Cashback (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Mickey's Once Upon a Christmas (1999)</td>\n",
       "      <td>Devil Dog: The Hound of Hell (1978)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Tokyo! (2008)</td>\n",
       "      <td>Mystery Train (1989)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>What Ever Happened to Baby Jane? (1962)</td>\n",
       "      <td>What Ever Happened to Baby Jane? (1962)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Meteor Man, The (1993)</td>\n",
       "      <td>Barefoot Contessa, The (1954)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Unknown White Male (2005)</td>\n",
       "      <td>Day Watch (Dnevnoy dozor) (2006)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Legend of 1900, The (a.k.a. The Legend of the ...</td>\n",
       "      <td>Fire and Ice (1983)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Goodbye Lover (1999)</td>\n",
       "      <td>Savages (2012)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Timecop (1994)</td>\n",
       "      <td>Tokyo! (2008)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>John Wick: Chapter Two (2017)</td>\n",
       "      <td>Meteor Man, The (1993)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Masterminds (1997)</td>\n",
       "      <td>Unknown White Male (2005)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Woodsman, The (2004)</td>\n",
       "      <td>Goodbye Lover (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Flipped (2010)</td>\n",
       "      <td>Masterminds (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Dragon Ball Z the Movie: The World's Strongest...</td>\n",
       "      <td>Woodsman, The (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Perfect Murder, A (1998)</td>\n",
       "      <td>Dragon Ball Z the Movie: The World's Strongest...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Comic-Con Episode IV: A Fan's Hope (2011)</td>\n",
       "      <td>Comic-Con Episode IV: A Fan's Hope (2011)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            Un-Ranked  \\\n",
       "0                             Flintstones, The (1994)   \n",
       "1                                Mystery Train (1989)   \n",
       "2                                     Cashback (2004)   \n",
       "3                    Day Watch (Dnevnoy dozor) (2006)   \n",
       "4                                          Big (1988)   \n",
       "5                                 Fire and Ice (1983)   \n",
       "6                                  Life Stinks (1991)   \n",
       "7                                      Savages (2012)   \n",
       "8                       Barefoot Contessa, The (1954)   \n",
       "9                 Devil Dog: The Hound of Hell (1978)   \n",
       "10              Mickey's Once Upon a Christmas (1999)   \n",
       "11                                      Tokyo! (2008)   \n",
       "12            What Ever Happened to Baby Jane? (1962)   \n",
       "13                             Meteor Man, The (1993)   \n",
       "14                          Unknown White Male (2005)   \n",
       "15  Legend of 1900, The (a.k.a. The Legend of the ...   \n",
       "16                               Goodbye Lover (1999)   \n",
       "17                                     Timecop (1994)   \n",
       "18                      John Wick: Chapter Two (2017)   \n",
       "19                                 Masterminds (1997)   \n",
       "20                               Woodsman, The (2004)   \n",
       "21                                     Flipped (2010)   \n",
       "22  Dragon Ball Z the Movie: The World's Strongest...   \n",
       "23                           Perfect Murder, A (1998)   \n",
       "24          Comic-Con Episode IV: A Fan's Hope (2011)   \n",
       "\n",
       "                                            Re-Ranked  \n",
       "0                             Flintstones, The (1994)  \n",
       "1                                          Big (1988)  \n",
       "2                                      Flipped (2010)  \n",
       "3                                  Life Stinks (1991)  \n",
       "4                                      Timecop (1994)  \n",
       "5   Legend of 1900, The (a.k.a. The Legend of the ...  \n",
       "6                       John Wick: Chapter Two (2017)  \n",
       "7                            Perfect Murder, A (1998)  \n",
       "8               Mickey's Once Upon a Christmas (1999)  \n",
       "9                                     Cashback (2004)  \n",
       "10                Devil Dog: The Hound of Hell (1978)  \n",
       "11                               Mystery Train (1989)  \n",
       "12            What Ever Happened to Baby Jane? (1962)  \n",
       "13                      Barefoot Contessa, The (1954)  \n",
       "14                   Day Watch (Dnevnoy dozor) (2006)  \n",
       "15                                Fire and Ice (1983)  \n",
       "16                                     Savages (2012)  \n",
       "17                                      Tokyo! (2008)  \n",
       "18                             Meteor Man, The (1993)  \n",
       "19                          Unknown White Male (2005)  \n",
       "20                               Goodbye Lover (1999)  \n",
       "21                                 Masterminds (1997)  \n",
       "22                               Woodsman, The (2004)  \n",
       "23  Dragon Ball Z the Movie: The World's Strongest...  \n",
       "24          Comic-Con Episode IV: A Fan's Hope (2011)  "
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ranked_list = []\n",
    "item_list = get_recommendations_response_rerank['personalizedRanking']\n",
    "for item in item_list:\n",
    "    movie = get_movie_by_id(item['itemId'])\n",
    "    ranked_list.append(movie)\n",
    "ranked_df = pd.DataFrame(ranked_list, columns = ['Re-Ranked'])\n",
    "rerank_df = pd.concat([rerank_df, ranked_df], axis=1)\n",
    "rerank_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see above how each entry was re-ordered based on the model's understanding of the user. This is a popular task when you have a collection of items to surface a user, a list of promotions for example."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Batch recommendations <a class=\"anchor\" id=\"batch\"></a>\n",
    "[Back to top](#top)\n",
    "\n",
    "There are many cases where you may want to have a larger dataset of exported recommendations. Recently, Amazon Personalize launched batch recommendations as a way to export a collection of recommendations to S3. In this example, we will walk through how to do this for the HRNN solution. For more information about batch recommendations, please see the [documentation](https://docs.aws.amazon.com/personalize/latest/dg/getting-recommendations.html#recommendations-batch). This feature applies to all recipes, but the output format will vary.\n",
    "\n",
    "A simple implementation looks like this:\n",
    "\n",
    "```python\n",
    "import boto3\n",
    "\n",
    "personalize_rec = boto3.client(service_name='personalize')\n",
    "\n",
    "personalize_rec.create_batch_inference_job (\n",
    "    solutionVersionArn = \"Solution version ARN\",\n",
    "    jobName = \"Batch job name\",\n",
    "    roleArn = \"IAM role ARN\",\n",
    "    jobInput = \n",
    "       {\"s3DataSource\": {\"path\": S3 input path}},\n",
    "    jobOutput = \n",
    "       {\"s3DataDestination\": {\"path\":S3 output path\"}}\n",
    ")\n",
    "```\n",
    "\n",
    "The SDK import, the solution version arn, and role arns have all been determined. This just leaves an input, an output, and a job name to be defined.\n",
    "\n",
    "Starting with the input for HRNN, it looks like:\n",
    "\n",
    "\n",
    "```JSON\n",
    "{\"userId\": \"4638\"}\n",
    "{\"userId\": \"663\"}\n",
    "{\"userId\": \"3384\"}\n",
    "```\n",
    "\n",
    "This should yield an output that looks like this:\n",
    "\n",
    "```JSON\n",
    "{\"input\":{\"userId\":\"4638\"}, \"output\": {\"recommendedItems\": [\"296\", \"1\", \"260\", \"318\"]}}\n",
    "{\"input\":{\"userId\":\"663\"}, \"output\": {\"recommendedItems\": [\"1393\", \"3793\", \"2701\", \"3826\"]}}\n",
    "{\"input\":{\"userId\":\"3384\"}, \"output\": {\"recommendedItems\": [\"8368\", \"5989\", \"40815\", \"48780\"]}}\n",
    "```\n",
    "\n",
    "The output is a JSON Lines file. It consists of individual JSON objects, one per line. So we will need to put in more work later to digest the results in this format."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Building the input file\n",
    "\n",
    "When you are using the batch feature, you specify the users that you'd like to receive recommendations for when the job has completed. The cell below will again select a few random users and will then build the file and save it to disk. From there, you will upload it to S3 to use in the API call later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We will use the same users from before\n",
    "users\n",
    "# Write the file to disk\n",
    "json_input_filename = \"json_input.json\"\n",
    "with open(data_dir + \"/\" + json_input_filename, 'w') as json_input:\n",
    "    for user_id in users:\n",
    "        json_input.write('{\"userId\": \"' + str(user_id) + '\"}\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"userId\": \"164\"}\r\n",
      "{\"userId\": \"563\"}\r\n",
      "{\"userId\": \"396\"}\r\n"
     ]
    }
   ],
   "source": [
    "# Showcase the input file:\n",
    "!cat $data_dir\"/\"$json_input_filename"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Upload the file to S3 and save the path as a variable for later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "s3://835319576252personalizepocvod/json_input.json\n"
     ]
    }
   ],
   "source": [
    "# Upload files to S3\n",
    "boto3.Session().resource('s3').Bucket(bucket_name).Object(json_input_filename).upload_file(data_dir+\"/\"+json_input_filename)\n",
    "s3_input_path = \"s3://\" + bucket_name + \"/\" + json_input_filename\n",
    "print(s3_input_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Batch recommendations read the input from the file we've uploaded to S3. Similarly, batch recommendations will save the output to file in S3. So we define the output path where the results should be saved."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "s3://835319576252personalizepocvod/\n"
     ]
    }
   ],
   "source": [
    "# Define the output path\n",
    "s3_output_path = \"s3://\" + bucket_name + \"/\"\n",
    "print(s3_output_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now just make the call to kick off the batch export process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "batchInferenceJobArn = personalize.create_batch_inference_job (\n",
    "    solutionVersionArn = userpersonalization_solution_version_arn,\n",
    "    jobName = \"VOD-POC-Batch-Inference-Job-UserPersonalization_\" + str(round(time.time()*1000)),\n",
    "    roleArn = role_arn,\n",
    "    jobInput = \n",
    "     {\"s3DataSource\": {\"path\": s3_input_path}},\n",
    "    jobOutput = \n",
    "     {\"s3DataDestination\":{\"path\": s3_output_path}}\n",
    ")\n",
    "batchInferenceJobArn = batchInferenceJobArn['batchInferenceJobArn']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the while loop below to track the status of the batch recommendation call. This can take around 30 minutes to complete, because Personalize needs to stand up the infrastructure to perform the task. We are testing the feature with a dataset of only 3 users, which is not an efficient use of this mechanism. Normally, you would only use this feature for bulk processing, in which case the efficiencies will become clear."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Import Started on:  08:37:36 PM\n",
      "DatasetInferenceJob: CREATE PENDING\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: CREATE IN_PROGRESS\n",
      "DatasetInferenceJob: ACTIVE\n",
      "Import Completed on:  09:03:39 PM\n",
      "CPU times: user 144 ms, sys: 34.2 ms, total: 178 ms\n",
      "Wall time: 26min 3s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "current_time = datetime.now()\n",
    "print(\"Import Started on: \", current_time.strftime(\"%I:%M:%S %p\"))\n",
    "\n",
    "max_time = time.time() + 6*60*60 # 6 hours\n",
    "while time.time() < max_time:\n",
    "    describe_dataset_inference_job_response = personalize.describe_batch_inference_job(\n",
    "        batchInferenceJobArn = batchInferenceJobArn\n",
    "    )\n",
    "    status = describe_dataset_inference_job_response[\"batchInferenceJob\"]['status']\n",
    "    print(\"DatasetInferenceJob: {}\".format(status))\n",
    "    \n",
    "    if status == \"ACTIVE\" or status == \"CREATE FAILED\":\n",
    "        break\n",
    "        \n",
    "    time.sleep(60)\n",
    "    \n",
    "current_time = datetime.now()\n",
    "print(\"Import Completed on: \", current_time.strftime(\"%I:%M:%S %p\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>User: 164</th>\n",
       "      <th>User: 563</th>\n",
       "      <th>User: 396</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Superman II (1980)</td>\n",
       "      <td>Fargo (1996)</td>\n",
       "      <td>Shrek (2001)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Star Trek II: The Wrath of Khan (1982)</td>\n",
       "      <td>Yes Man (2008)</td>\n",
       "      <td>Amelie (Fabuleux destin d'AmÃ©lie Poulain, Le)...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Running Man, The (1987)</td>\n",
       "      <td>Holiday, The (2006)</td>\n",
       "      <td>Lord of the Rings: The Two Towers, The (2002)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Star Trek V: The Final Frontier (1989)</td>\n",
       "      <td>Borat: Cultural Learnings of America for Make ...</td>\n",
       "      <td>Toy Story 2 (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Planet of the Apes (1968)</td>\n",
       "      <td>Devil Wears Prada, The (2006)</td>\n",
       "      <td>Good Will Hunting (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Star Trek VI: The Undiscovered Country (1991)</td>\n",
       "      <td>Notting Hill (1999)</td>\n",
       "      <td>Eternal Sunshine of the Spotless Mind (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Terminator, The (1984)</td>\n",
       "      <td>The Imitation Game (2014)</td>\n",
       "      <td>Spirited Away (Sen to Chihiro no kamikakushi) ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Tron (1982)</td>\n",
       "      <td>Hurt Locker, The (2008)</td>\n",
       "      <td>Lord of the Rings: The Return of the King, The...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Star Trek III: The Search for Spock (1984)</td>\n",
       "      <td>Hitch (2005)</td>\n",
       "      <td>Schindler's List (1993)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>RoboCop (1987)</td>\n",
       "      <td>Mean Girls (2004)</td>\n",
       "      <td>LÃ©on: The Professional (a.k.a. The Profession...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Abyss, The (1989)</td>\n",
       "      <td>Forgetting Sarah Marshall (2008)</td>\n",
       "      <td>Beautiful Mind, A (2001)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Superman (1978)</td>\n",
       "      <td>In Bruges (2008)</td>\n",
       "      <td>Monsters, Inc. (2001)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Rocketeer, The (1991)</td>\n",
       "      <td>Hangover, The (2009)</td>\n",
       "      <td>Life Is Beautiful (La Vita Ã¨ bella) (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Total Recall (1990)</td>\n",
       "      <td>Wedding Crashers (2005)</td>\n",
       "      <td>Forrest Gump (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Star Trek IV: The Voyage Home (1986)</td>\n",
       "      <td>(500) Days of Summer (2009)</td>\n",
       "      <td>Godfather, The (1972)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Star Trek: The Motion Picture (1979)</td>\n",
       "      <td>Zombieland (2009)</td>\n",
       "      <td>Finding Nemo (2003)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Logan's Run (1976)</td>\n",
       "      <td>Up in the Air (2009)</td>\n",
       "      <td>Lion King, The (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Top Gun (1986)</td>\n",
       "      <td>Girl with the Dragon Tattoo, The (MÃ¤n som hat...</td>\n",
       "      <td>American Beauty (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Romancing the Stone (1984)</td>\n",
       "      <td>Proposal, The (2009)</td>\n",
       "      <td>Lord of the Rings: The Fellowship of the Ring,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Back to the Future Part III (1990)</td>\n",
       "      <td>Life Is Beautiful (La Vita Ã¨ bella) (1997)</td>\n",
       "      <td>2001: A Space Odyssey (1968)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Escape from New York (1981)</td>\n",
       "      <td>Knocked Up (2007)</td>\n",
       "      <td>Pirates of the Caribbean: The Curse of the Bla...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Road Warrior, The (Mad Max 2) (1981)</td>\n",
       "      <td>Deadpool (2016)</td>\n",
       "      <td>Apollo 13 (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Thing, The (1982)</td>\n",
       "      <td>Prestige, The (2006)</td>\n",
       "      <td>Pulp Fiction (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Willow (1988)</td>\n",
       "      <td>Idiocracy (2006)</td>\n",
       "      <td>Harry Potter and the Prisoner of Azkaban (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Ghostbusters (a.k.a. Ghost Busters) (1984)</td>\n",
       "      <td>Anchorman: The Legend of Ron Burgundy (2004)</td>\n",
       "      <td>City of God (Cidade de Deus) (2002)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                        User: 164  \\\n",
       "0                              Superman II (1980)   \n",
       "1          Star Trek II: The Wrath of Khan (1982)   \n",
       "2                         Running Man, The (1987)   \n",
       "3          Star Trek V: The Final Frontier (1989)   \n",
       "4                       Planet of the Apes (1968)   \n",
       "5   Star Trek VI: The Undiscovered Country (1991)   \n",
       "6                          Terminator, The (1984)   \n",
       "7                                     Tron (1982)   \n",
       "8      Star Trek III: The Search for Spock (1984)   \n",
       "9                                  RoboCop (1987)   \n",
       "10                              Abyss, The (1989)   \n",
       "11                                Superman (1978)   \n",
       "12                          Rocketeer, The (1991)   \n",
       "13                            Total Recall (1990)   \n",
       "14           Star Trek IV: The Voyage Home (1986)   \n",
       "15           Star Trek: The Motion Picture (1979)   \n",
       "16                             Logan's Run (1976)   \n",
       "17                                 Top Gun (1986)   \n",
       "18                     Romancing the Stone (1984)   \n",
       "19             Back to the Future Part III (1990)   \n",
       "20                    Escape from New York (1981)   \n",
       "21           Road Warrior, The (Mad Max 2) (1981)   \n",
       "22                              Thing, The (1982)   \n",
       "23                                  Willow (1988)   \n",
       "24     Ghostbusters (a.k.a. Ghost Busters) (1984)   \n",
       "\n",
       "                                            User: 563  \\\n",
       "0                                        Fargo (1996)   \n",
       "1                                      Yes Man (2008)   \n",
       "2                                 Holiday, The (2006)   \n",
       "3   Borat: Cultural Learnings of America for Make ...   \n",
       "4                       Devil Wears Prada, The (2006)   \n",
       "5                                 Notting Hill (1999)   \n",
       "6                           The Imitation Game (2014)   \n",
       "7                             Hurt Locker, The (2008)   \n",
       "8                                        Hitch (2005)   \n",
       "9                                   Mean Girls (2004)   \n",
       "10                   Forgetting Sarah Marshall (2008)   \n",
       "11                                   In Bruges (2008)   \n",
       "12                               Hangover, The (2009)   \n",
       "13                            Wedding Crashers (2005)   \n",
       "14                        (500) Days of Summer (2009)   \n",
       "15                                  Zombieland (2009)   \n",
       "16                               Up in the Air (2009)   \n",
       "17  Girl with the Dragon Tattoo, The (MÃ¤n som hat...   \n",
       "18                               Proposal, The (2009)   \n",
       "19        Life Is Beautiful (La Vita Ã¨ bella) (1997)   \n",
       "20                                  Knocked Up (2007)   \n",
       "21                                    Deadpool (2016)   \n",
       "22                               Prestige, The (2006)   \n",
       "23                                   Idiocracy (2006)   \n",
       "24       Anchorman: The Legend of Ron Burgundy (2004)   \n",
       "\n",
       "                                            User: 396  \n",
       "0                                        Shrek (2001)  \n",
       "1   Amelie (Fabuleux destin d'AmÃ©lie Poulain, Le)...  \n",
       "2       Lord of the Rings: The Two Towers, The (2002)  \n",
       "3                                  Toy Story 2 (1999)  \n",
       "4                            Good Will Hunting (1997)  \n",
       "5        Eternal Sunshine of the Spotless Mind (2004)  \n",
       "6   Spirited Away (Sen to Chihiro no kamikakushi) ...  \n",
       "7   Lord of the Rings: The Return of the King, The...  \n",
       "8                             Schindler's List (1993)  \n",
       "9   LÃ©on: The Professional (a.k.a. The Profession...  \n",
       "10                           Beautiful Mind, A (2001)  \n",
       "11                              Monsters, Inc. (2001)  \n",
       "12        Life Is Beautiful (La Vita Ã¨ bella) (1997)  \n",
       "13                                Forrest Gump (1994)  \n",
       "14                              Godfather, The (1972)  \n",
       "15                                Finding Nemo (2003)  \n",
       "16                              Lion King, The (1994)  \n",
       "17                             American Beauty (1999)  \n",
       "18  Lord of the Rings: The Fellowship of the Ring,...  \n",
       "19                       2001: A Space Odyssey (1968)  \n",
       "20  Pirates of the Caribbean: The Curse of the Bla...  \n",
       "21                                   Apollo 13 (1995)  \n",
       "22                                Pulp Fiction (1994)  \n",
       "23    Harry Potter and the Prisoner of Azkaban (2004)  \n",
       "24                City of God (Cidade de Deus) (2002)  "
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s3 = boto3.client('s3')\n",
    "export_name = json_input_filename + \".out\"\n",
    "s3.download_file(bucket_name, export_name, data_dir+\"/\"+export_name)\n",
    "\n",
    "# Update DF rendering\n",
    "pd.set_option('display.max_rows', 30)\n",
    "with open(data_dir+\"/\"+export_name) as json_file:\n",
    "    # Get the first line and parse it\n",
    "    line = json.loads(json_file.readline())\n",
    "    # Do the same for the other lines\n",
    "    while line:\n",
    "        # extract the user ID \n",
    "        col_header = \"User: \" + line['input']['userId']\n",
    "        # Create a list for all the artists\n",
    "        recommendation_list = []\n",
    "        # Add all the entries\n",
    "        for item in line['output']['recommendedItems']:\n",
    "            movie = get_movie_by_id(item)\n",
    "            recommendation_list.append(movie)\n",
    "        if 'bulk_recommendations_df' in locals():\n",
    "            new_rec_DF = pd.DataFrame(recommendation_list, columns = [col_header])\n",
    "            bulk_recommendations_df = bulk_recommendations_df.join(new_rec_DF)\n",
    "        else:\n",
    "            bulk_recommendations_df = pd.DataFrame(recommendation_list, columns=[col_header])\n",
    "        try:\n",
    "            line = json.loads(json_file.readline())\n",
    "        except:\n",
    "            line = None\n",
    "bulk_recommendations_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Wrap up <a class=\"anchor\" id=\"wrapup\"></a>\n",
    "[Back to top](#top)\n",
    "\n",
    "With that you now have a fully working collection of models to tackle various recommendation and personalization scenarios, as well as the skills to manipulate customer data to better integrate with the service, and a knowledge of how to do all this over APIs and by leveraging open source data science tools.\n",
    "\n",
    "Use these notebooks as a guide to getting started with your customers for POCs. As you find missing components, or discover new approaches, cut a pull request and provide any additional helpful components that may be missing from this collection.\n",
    "\n",
    "You'll want to make sure that you clean up all of the resources deployed during this POC. We have provided a separate notebook which shows you how to identify and delete the resources in `04_Clean_Up_Resources.ipynb`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stored 'event_tracker_arn' (str)\n",
      "Stored 'batchInferenceJobArn' (str)\n"
     ]
    }
   ],
   "source": [
    "%store event_tracker_arn\n",
    "%store batchInferenceJobArn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Release Resources"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%javascript\n",
    "Jupyter.notebook.save_checkpoint();\n",
    "Jupyter.notebook.session.delete();"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_python3",
   "language": "python",
   "name": "conda_python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
